-- Logs begin at Thu 2022-07-14 19:35:54 UTC, end at Thu 2022-07-14 20:52:50 UTC. -- Jul 14 20:19:24.086449 dv-cinder-799871-32-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 14 20:19:24.095975 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 14 20:19:24.097993 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 14 20:19:24 2022] *** Jul 14 20:19:24.097993 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 14 20:19:24.097993 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 14 20:19:24.097993 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: nodename: dv-cinder-799871-32-iscsi Jul 14 20:19:24.097993 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: machine: x86_64 Jul 14 20:19:24.097993 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: clock source: unix Jul 14 20:19:24.097993 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: pcre jit disabled Jul 14 20:19:24.097993 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: detected number of CPU cores: 4 Jul 14 20:19:24.097993 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: current working directory: / Jul 14 20:19:24.097993 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: detected binary path: /usr/bin/uwsgi-core Jul 14 20:19:24.098541 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: your processes number limit is 47680 Jul 14 20:19:24.098541 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: your memory page size is 4096 bytes Jul 14 20:19:24.098541 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: detected max file descriptor number: 2048 Jul 14 20:19:24.098541 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: lock engine: pthread robust mutexes Jul 14 20:19:24.098541 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: thunder lock: enabled Jul 14 20:19:24.098541 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 14 20:19:24.098541 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 14 20:19:24.137662 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: Python main interpreter initialized at 0x560f8219e9c0 Jul 14 20:19:24.137662 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: python threads support enabled Jul 14 20:19:24.137662 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: your server socket listen backlog is limited to 100 connections Jul 14 20:19:24.137662 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: your mercy for graceful operations on workers is 90 seconds Jul 14 20:19:24.138022 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: mapped 403077 bytes (393 KB) for 2 cores Jul 14 20:19:24.138102 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: *** Operational MODE: preforking *** Jul 14 20:19:24.138351 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: *** uWSGI is running in multiple interpreter mode *** Jul 14 20:19:24.138351 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: spawned uWSGI master process (pid: 80169) Jul 14 20:19:24.138554 dv-cinder-799871-32-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 14 20:19:24.142963 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: spawned uWSGI worker 1 (pid: 80177, cores: 1) Jul 14 20:19:24.142963 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: spawned uWSGI worker 2 (pid: 80178, cores: 1) Jul 14 20:19:24.142963 dv-cinder-799871-32-iscsi devstack@keystone.service[80169]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 14 20:19:29.597621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 20:19:29.597621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: from cryptography.utils import int_from_bytes Jul 14 20:19:29.598425 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 20:19:29.598425 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: from cryptography.utils import int_from_bytes Jul 14 20:19:29.600363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 20:19:29.600363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: from cryptography.utils import int_from_bytes Jul 14 20:19:29.600813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 20:19:29.600813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: from cryptography.utils import int_from_bytes Jul 14 20:19:29.647007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 20:19:29.647202 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 20:19:29.647398 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] command line args: [] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 20:19:29.647668 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 20:19:29.647841 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ================================================================================ {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 20:19:29.648094 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] admin_token = **** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.648226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 20:19:29.648363 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] config_dir = [] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.648498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 20:19:29.648633 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.648758 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] command line args: [] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 20:19:29.648884 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] config_source = [] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.648968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 20:19:29.649044 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] debug = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.649113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ================================================================================ {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 20:19:29.649179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.649464 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] admin_token = **** {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.649535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.649593 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] config_dir = [] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.649648 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.649708 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.649775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.649851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] config_source = [] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.649962 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] debug = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.650028 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.650182 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.650288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] list_limit = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.650413 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] log_config_append = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.650524 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.650617 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.650685 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.650743 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] log_dir = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.650807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.650870 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] log_file = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.651057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.651171 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.651278 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] list_limit = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.651341 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.651475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] log_config_append = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.651587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.651713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.651788 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] log_dir = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.651901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.651976 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] log_file = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.652099 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.652175 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.652288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.652366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.652518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.652604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.652682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.652829 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.652911 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.653061 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.653137 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.653263 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.653343 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.653503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.653627 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.653746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.653827 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.653981 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.654062 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.654217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.654302 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.654452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.654686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.654776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.654889 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] publish_errors = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.655031 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.655679 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.655764 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.655911 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.655986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.656109 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.656317 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.656486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.656573 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.656671 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] publish_errors = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.656753 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] standard_threads = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.656906 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.656998 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.657137 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.657213 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.657332 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.657460 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.657539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] use_journal = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.657607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.657674 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] use_json = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.657770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.657852 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] use_stderr = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.658003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] use_syslog = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.658079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] standard_threads = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.658233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.658309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.658486 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.658562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.658666 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.658741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.658845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] use_journal = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.658985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.659065 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] use_json = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.659213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] use_stderr = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.659300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.659392 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] use_syslog = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.659467 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.659625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 20:19:29.659699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.659886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.659972 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.660130 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.660204 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.660374 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.660449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.660616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.660702 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.660864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.660944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.661093 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.661167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.661337 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.661425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.661571 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.661646 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.661813 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.661889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.662058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.662144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.662288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.662365 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.662529 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.662603 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.662774 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.662923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.663036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.663110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.663268 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.663342 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.663517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.663603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.663752 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.663826 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.664007 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.664082 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.664249 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.664334 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.664476 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.664557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.664702 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.664784 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.664929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.665006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.665162 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.665237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.665407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.665488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.665651 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.665727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.665877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.665951 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.666104 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.666184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.666342 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.666417 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.666582 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.666655 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.666817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.666891 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.667083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.667168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.667312 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.667387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.667541 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.667620 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.667769 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.667844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.667999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.668073 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.668228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.668302 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.668448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.668527 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.668671 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.668748 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.668906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.668980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.669149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.669225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.669375 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.669450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.669605 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.669686 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.669831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.669907 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.670063 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.670164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.670294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.670400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.670540 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.670642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.670841 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.671468 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.671614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.671748 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.671897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.671972 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.672091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.672164 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.672276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.672380 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.672495 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.672579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.672723 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.672808 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.672964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.673038 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.673195 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.673274 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.673430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.673545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.673656 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.673741 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.673896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.673983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.674130 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.674214 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.674357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.674434 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.674592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.674683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.674827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.674904 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.675099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.675223 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.675344 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.675426 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.675586 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.675662 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.675822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.675901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.676064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.676154 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.676304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.676380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.676575 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.676654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.676815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.676901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.677068 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.677144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.677314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.677390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.677554 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.677641 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.677792 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.677868 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.678031 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.678111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.678282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.678369 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.678522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.678602 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.678761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.678839 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.679029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.679122 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.679271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.679351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.679509 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.679588 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.679752 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.679839 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.679990 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.680110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.680226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.680452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.680527 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.680699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.680779 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.680936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.681012 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.681166 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.681242 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.681395 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.681481 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.681625 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.681708 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.681859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.681936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.682088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.682164 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.682314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.682391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.682546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.682621 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.682776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.682859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.683034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.683122 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.683289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.683372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.683527 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.683670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.683767 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.683915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.684017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.684150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] auth.external = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.684256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.684383 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.684514 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.684680 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.684759 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.684915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.684991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.685147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] auth.password = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.685252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] auth.external = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.685378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] auth.token = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.685457 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.685617 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.685718 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.685855 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.685953 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.686100 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.686178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] auth.password = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.686346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.686422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] auth.token = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.686602 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.686681 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.686851 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.686977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.687105 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.687275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.687351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] credential.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.687517 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.687605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.687772 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.687851 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.688026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.688103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.688262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.688350 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.688502 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.688578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] credential.caching = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.688737 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.688813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.688991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.689113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.689241 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.689317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.689479 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.689569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.689731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.689815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.689974 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.690130 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.690240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.690391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.690497 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.690675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.690754 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.690920 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.691016 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.691198 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.691277 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.691436 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] federation.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.691514 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.691673 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.691771 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.691917 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.692018 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.692169 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.692278 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.692544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.692690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] federation.caching = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.692806 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.692882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.692962 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.693034 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.693216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.693292 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.693449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.693526 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.693701 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.693780 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.693935 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.694012 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.694182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.694267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.694419 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.694501 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.694664 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.694740 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.694902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.695120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.695201 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.695316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.695391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.695506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] identity.caching = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.695620 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.695706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.695849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.695927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.696087 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.696295 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.696373 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.696554 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.696633 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.696808 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.696889 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.697075 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.697152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.697294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.697444 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.697558 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.697706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.697806 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.697965 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.698050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.698201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.698278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.698435 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.698511 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.698669 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.698780 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.698920 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.699054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.699190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.699307 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.699440 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.699540 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.699698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.699788 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.699928 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.700008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.700168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.700288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.700401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.700481 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.700638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.700717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.700887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.700976 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.701113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.701192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.701349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.701427 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.701589 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.701681 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.701822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.701903 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.702592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.702685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.702835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.702926 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.703088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.703168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.703326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.703405 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.703553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.703645 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.703788 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.703868 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.704023 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.704101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.704264 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.704353 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.704494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.704579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.704725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.704807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.704955 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.705051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.705194 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.705273 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.705418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.705495 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.705645 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.705733 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.705890 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.705967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.706139 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.706217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.706379 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.706477 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.706617 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.706697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.706853 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.706977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.707102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.707194 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.707342 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.707426 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.707590 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.707668 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.707815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.707895 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.708057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.708141 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.708296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.708375 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.708528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.708609 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.708758 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.708835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.708986 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.709068 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.709220 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.709301 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.709452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.709531 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.709685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.709764 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.709912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.709994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.710154 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.710232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.710390 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.710473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.710628 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.710710 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.710865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.710963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.711128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.711206 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.711358 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.711438 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.711599 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.711677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.711833 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.711911 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.712071 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.712196 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.712301 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.712388 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.712567 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.712690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.712801 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.712876 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.712995 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.713075 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.713230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.713322 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.713462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.713541 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.713688 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.713766 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.713923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.714001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.714152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.714232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.714386 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.714472 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.714617 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.714704 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.714858 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.714987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.715114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.715192 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.715361 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.715438 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.715596 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.715677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.715833 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] resource.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.715911 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.716070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.716149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.716312 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.716392 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.716546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.716625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.716778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.716855 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.717016 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.717097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.717248 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.717326 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] resource.caching = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.717490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.717569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.718310 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.718420 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.718547 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.718628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.718798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.718884 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] role.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.719085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] role.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.719165 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.719288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.719375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.719537 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.719616 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.719780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.719859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.720022 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.720101 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.720262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] role.caching = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.720349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.720503 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.720582 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] role.driver = sql {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.720737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.720814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.720978 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.721064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.721222 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.721302 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.721456 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.721535 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.721700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.721787 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.721944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.722023 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.722187 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.722267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.722432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.722530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.722679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.722856 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.722973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.723113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.723193 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.723361 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.723449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.723605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.723685 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.723849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.723929 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.724094 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.724287 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.724367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.724526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.724604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.724778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.724869 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.725021 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.725099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.725253 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.725339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.725483 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.725571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.725713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.725833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.725961 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.726038 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.726213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.726292 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.726460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.726547 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.726690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.726767 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.726921 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.727050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.727171 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.727248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.727407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] token.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.727484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.727642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.727723 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.727887 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.727965 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.728121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.728200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.728357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.728437 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.728592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.728670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.728829 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.728906 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] token.caching = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.729065 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.729144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.729303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.729380 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.729540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.729617 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.729772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.729851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.730008 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.730087 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.730241 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.730319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.730484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.730565 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.730728 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.730806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.730975 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.731058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.731219 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.731301 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.731475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.731552 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.731716 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.731795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.connection = **** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.731972 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.732111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.732225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.732338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.732456 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.732567 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.732675 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.732800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.732918 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.733037 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.733151 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.733278 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.connection = **** {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.733421 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.733561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.733694 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.733831 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.733958 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.734082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.734203 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.734323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.734448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.734584 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.734708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.734830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.734970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.735094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.735215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.735354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.735483 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.735604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.735725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.735847 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.735969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.736106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.736228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.736350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.736474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80177) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 20:19:29.736597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.736597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.736829 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.736951 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.737120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.737359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 20:19:29.737530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80178) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 20:19:29.766884 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:29.767139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:29.767613 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:29.767918 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:29.768155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:29.768384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:29.775951 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:29.776751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:29.804694 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:29.804925 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:29.805261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:29.805501 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:29.948086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:29.948265 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:29.997393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:29.997619 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:29.998083 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:29.998326 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.063455 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.063640 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.063744 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.063847 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.063927 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.064032 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.064115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.064255 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.064336 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.064475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.064559 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.064693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.089915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.090142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.090251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.090367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.090464 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.090587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.090670 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.090805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.090889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.091044 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.091128 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.091259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.091343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.091475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.102276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.102598 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.114949 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.115187 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.115288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.115413 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.115497 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.115634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.115717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.115852 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.115936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.116065 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.116148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.116279 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80177) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.116368 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.116513 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80178) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 20:19:30.134066 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.134277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 20:19:30.134361 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.134577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 20:19:30.134674 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.134817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 20:19:30.134980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.135126 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 20:19:30.135335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.135597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.135682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.135859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.135941 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.136115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.136195 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.136375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.147740 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.148012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.148107 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.148272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.148380 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.148541 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.148632 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.148865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.156757 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.157161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.160036 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.160313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.160497 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.160636 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.160776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.160903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.161058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.161146 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.161322 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.161404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.161599 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.161681 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.161859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.161991 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.162123 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.162251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.162395 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.162524 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.162647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.162774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.162902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.163187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.189621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.190301 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.191257 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.191526 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.191949 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.192232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.196776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.197051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.197327 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.197540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.197820 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.198146 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.205310 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.205588 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.206129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.206409 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.210058 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.210341 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.210616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.210890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.211045 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.211170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.211303 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.211427 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.211549 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.211680 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.211805 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.211928 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.212050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.212275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.212547 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.212817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.230611 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.230883 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.231611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.231889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.235449 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.236533 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.239104 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.239394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.240355 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.240659 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.244520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.245820 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.248191 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.250018 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.252398 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.252684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.252975 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.253237 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.253524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.253696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.253813 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.253996 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.254286 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.254568 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.254852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.255177 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.273688 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.273966 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.274228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.274490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.275147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.275436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.275708 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.275997 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.282404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.284007 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.284134 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.284404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.285731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.286005 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.287587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.287867 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.288154 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.288448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.288729 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.289217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.289503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.289786 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.290074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.290364 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.302040 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.302312 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.302587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.302861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.303162 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.303448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.303718 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.303844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.304046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.304333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.304611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.304889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.305172 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.305453 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.323848 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.325967 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.327751 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.329851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.331070 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.332666 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.333213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.334433 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.334707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.334862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.335022 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.336624 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.336906 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.337193 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.342254 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.345247 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.345417 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.346270 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.347201 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.347487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.347764 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.348034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.350817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.351140 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.351428 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.351703 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.358994 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.359284 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.359558 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.366431 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.366727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.367036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.367869 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80177) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.368146 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.368426 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.368692 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.368964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.369242 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.369525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.369814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.370133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.370486 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.370655 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.370928 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.371211 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.371469 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80177) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.375497 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80178) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 20:19:30.375781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.376061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.376338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.376607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.376898 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.377195 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.377456 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.377827 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.378099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.378385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.378660 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.378930 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.379224 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80178) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 20:19:30.410480 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560f8219e9c0 pid: 80177 (default app) Jul 14 20:19:30.414017 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9b726-7a8e-4483-8ebf-0b5fb3722ee2 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:30.414311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9b726-7a8e-4483-8ebf-0b5fb3722ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:30.414581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9b726-7a8e-4483-8ebf-0b5fb3722ee2 None None] PATH_INFO: `/v3` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:30.417996 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560f8219e9c0 pid: 80178 (default app) Jul 14 20:19:30.418789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Thu Jul 14 20:19:30 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:38.144909 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e9eaa-1039-4ba6-a514-61e0b0c3717d None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:38.145475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e9eaa-1039-4ba6-a514-61e0b0c3717d None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:38.145475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e9eaa-1039-4ba6-a514-61e0b0c3717d None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:38.149755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:19:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:19:38.155902 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d3c85-7367-4682-b457-66d4d9c6cc3d None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:38.156339 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d3c85-7367-4682-b457-66d4d9c6cc3d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:38.156457 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d3c85-7367-4682-b457-66d4d9c6cc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:38.224526 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: INFO dbcounter [None req-e44d3c85-7367-4682-b457-66d4d9c6cc3d None None] Registered counter for database keystone Jul 14 20:19:38.245860 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG oslo_db.sqlalchemy.engines [None req-e44d3c85-7367-4682-b457-66d4d9c6cc3d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80177) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 20:19:38.252249 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writer thread running {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 20:19:38.430421 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG passlib.handlers.bcrypt [None req-e44d3c85-7367-4682-b457-66d4d9c6cc3d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80177) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 20:19:38.430764 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG passlib.handlers.bcrypt [None req-e44d3c85-7367-4682-b457-66d4d9c6cc3d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80177) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 20:19:38.486118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e44d3c85-7367-4682-b457-66d4d9c6cc3d None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:19:38.487704 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e44d3c85-7367-4682-b457-66d4d9c6cc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:19:38.614372 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:19:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 14 20:19:38.797210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e9eaa-1039-4ba6-a514-61e0b0c3717d None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:38.865749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: INFO dbcounter [None req-733e9eaa-1039-4ba6-a514-61e0b0c3717d None None] Registered counter for database keystone Jul 14 20:19:38.885023 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG oslo_db.sqlalchemy.engines [None req-733e9eaa-1039-4ba6-a514-61e0b0c3717d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80178) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 20:19:38.891900 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writer thread running {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 20:19:39.011806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e9eaa-1039-4ba6-a514-61e0b0c3717d None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:39.016752 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4fc7b-7290-482a-8bab-b040202b6b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:39.017428 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4fc7b-7290-482a-8bab-b040202b6b28 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:39.017974 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4fc7b-7290-482a-8bab-b040202b6b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:39.018255 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4fc7b-7290-482a-8bab-b040202b6b28 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:39.027034 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4fc7b-7290-482a-8bab-b040202b6b28 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:39.079872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: WARNING keystone.common.rbac_enforcer.enforcer [None req-55c4fc7b-7290-482a-8bab-b040202b6b28 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 20:19:39.080132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4fc7b-7290-482a-8bab-b040202b6b28 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:39.090682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Thu Jul 14 20:19:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 20:19:39.099560 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d3c85-7367-4682-b457-66d4d9c6cc3d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:39.114395 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d3c85-7367-4682-b457-66d4d9c6cc3d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:39.118814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be96a4-11ff-4241-b503-690bc8bcbb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:39.119958 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be96a4-11ff-4241-b503-690bc8bcbb1b admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:39.120243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be96a4-11ff-4241-b503-690bc8bcbb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:39.120515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be96a4-11ff-4241-b503-690bc8bcbb1b admin admin] PATH_INFO: `/v3/projects/a9054cd3a81847fdaa8f38a717dc0938` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:39.122276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be96a4-11ff-4241-b503-690bc8bcbb1b admin admin] RBAC: Authorizing `identity:get_project(project_id=a9054cd3a81847fdaa8f38a717dc0938)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:39.175973 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d2be96a4-11ff-4241-b503-690bc8bcbb1b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 20:19:39.176626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be96a4-11ff-4241-b503-690bc8bcbb1b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:39.181034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Thu Jul 14 20:19:39 2022] GET /identity/v3/projects/a9054cd3a81847fdaa8f38a717dc0938 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:41.210158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4a773-6735-43f6-979c-a11c10adc299 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:41.210689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4a773-6735-43f6-979c-a11c10adc299 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:41.210689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4a773-6735-43f6-979c-a11c10adc299 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:41.214787 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:19:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:19:41.221009 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a6a6a-c864-47c8-b417-dad1dbdd205b None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:41.221430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a6a6a-c864-47c8-b417-dad1dbdd205b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:41.221563 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a6a6a-c864-47c8-b417-dad1dbdd205b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:41.240639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-0c7a6a6a-c864-47c8-b417-dad1dbdd205b None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:19:41.242459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0c7a6a6a-c864-47c8-b417-dad1dbdd205b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:19:41.250775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:19:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 14 20:19:41.273181 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4a773-6735-43f6-979c-a11c10adc299 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:41.286663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4a773-6735-43f6-979c-a11c10adc299 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:41.290805 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431235c-f27b-4d2d-be49-cd751580b069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:41.291462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431235c-f27b-4d2d-be49-cd751580b069 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:41.291749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431235c-f27b-4d2d-be49-cd751580b069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:41.292021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431235c-f27b-4d2d-be49-cd751580b069 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:41.299818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431235c-f27b-4d2d-be49-cd751580b069 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:41.301574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431235c-f27b-4d2d-be49-cd751580b069 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:41.310351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Thu Jul 14 20:19:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 20:19:41.316680 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a6a6a-c864-47c8-b417-dad1dbdd205b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:41.327065 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a6a6a-c864-47c8-b417-dad1dbdd205b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:41.332122 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93faa7-61e7-4c88-9011-6a00362c15a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:41.332986 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93faa7-61e7-4c88-9011-6a00362c15a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:41.333272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93faa7-61e7-4c88-9011-6a00362c15a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:41.333551 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93faa7-61e7-4c88-9011-6a00362c15a8 admin admin] PATH_INFO: `/v3/users/8b5c1280f1184855bd78412238939419` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:41.334966 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93faa7-61e7-4c88-9011-6a00362c15a8 admin admin] RBAC: Authorizing `identity:get_user(user_id=8b5c1280f1184855bd78412238939419)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:41.336835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93faa7-61e7-4c88-9011-6a00362c15a8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:41.338417 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Thu Jul 14 20:19:41 2022] GET /identity/v3/users/8b5c1280f1184855bd78412238939419 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:43.452299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1d043-fc11-4660-b301-7cfe11d24327 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:43.452815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1d043-fc11-4660-b301-7cfe11d24327 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:43.452920 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1d043-fc11-4660-b301-7cfe11d24327 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:43.454207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:19:43.459876 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae508d-4870-4545-b9c3-9d7df811711b None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:43.460428 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae508d-4870-4545-b9c3-9d7df811711b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:43.460428 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae508d-4870-4545-b9c3-9d7df811711b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:43.477327 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efbe77-b71b-40ca-b2ef-9dd19e8e281c None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:43.477534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-77ae508d-4870-4545-b9c3-9d7df811711b None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:19:43.477699 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efbe77-b71b-40ca-b2ef-9dd19e8e281c None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:43.477863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efbe77-b71b-40ca-b2ef-9dd19e8e281c None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:43.478548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-77ae508d-4870-4545-b9c3-9d7df811711b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:19:43.479237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:19:43.484785 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1da7d3-e2d8-472d-8987-3bd2a34a757a None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:43.485068 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1da7d3-e2d8-472d-8987-3bd2a34a757a None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:43.485344 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1da7d3-e2d8-472d-8987-3bd2a34a757a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:43.526350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG passlib.handlers.bcrypt [None req-4a1da7d3-e2d8-472d-8987-3bd2a34a757a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80178) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 20:19:43.526775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG passlib.handlers.bcrypt [None req-4a1da7d3-e2d8-472d-8987-3bd2a34a757a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80178) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 20:19:43.563200 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:19:43.572785 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-4a1da7d3-e2d8-472d-8987-3bd2a34a757a None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:19:43.574338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4a1da7d3-e2d8-472d-8987-3bd2a34a757a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:19:43.585963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae508d-4870-4545-b9c3-9d7df811711b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:43.611198 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:43.678431 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:19:43.696696 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae508d-4870-4545-b9c3-9d7df811711b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:43.700575 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1da7d3-e2d8-472d-8987-3bd2a34a757a None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:43.731270 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc319cc-d9c6-45cc-bdb1-ba98e37d17dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:43.731995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc319cc-d9c6-45cc-bdb1-ba98e37d17dc None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:43.732283 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc319cc-d9c6-45cc-bdb1-ba98e37d17dc None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:43.732568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc319cc-d9c6-45cc-bdb1-ba98e37d17dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:43.744499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc319cc-d9c6-45cc-bdb1-ba98e37d17dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:19:43.755761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-fdc319cc-d9c6-45cc-bdb1-ba98e37d17dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:19:43.755761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:19:43.755761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:19:43.755761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:19:43.755761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:19:43.755761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:19:43.755761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:19:43.755761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:19:43.755761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:19:43.755761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:19:43.755761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:19:43.755761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:19:43.755761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:19:43.755761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:19:43.755761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:19:43.755761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:19:43.756854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:19:43.756854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:19:43.756854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:19:43.756854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:19:43.756854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:19:43.756854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:19:43.756854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:19:43.756854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:19:43.756854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:19:43.756854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:19:43.756854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:19:43.756854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:19:43.756854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:19:43.756854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:19:43.756854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:19:43.756854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:19:43.757789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:19:43.757789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:19:43.757789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:19:43.757789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:19:43.757789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:19:43.757789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:19:43.757789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:19:43.757789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:19:43.757789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:19:43.757789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:19:43.757789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:19:43.757789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:19:43.757789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:19:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:19:43.762897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc319cc-d9c6-45cc-bdb1-ba98e37d17dc None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:43.805157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1da7d3-e2d8-472d-8987-3bd2a34a757a None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:43.807942 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:43.843951 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad88ad-54a3-413b-a281-18f09c2aec06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:43.844866 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad88ad-54a3-413b-a281-18f09c2aec06 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:43.845146 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad88ad-54a3-413b-a281-18f09c2aec06 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:43.845423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad88ad-54a3-413b-a281-18f09c2aec06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:43.851595 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:43.870801 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc319cc-d9c6-45cc-bdb1-ba98e37d17dc None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:43.906908 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a0898-9994-4500-b004-fbc2152d7dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:43.907610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a0898-9994-4500-b004-fbc2152d7dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:43.907897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a0898-9994-4500-b004-fbc2152d7dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:43.908171 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a0898-9994-4500-b004-fbc2152d7dc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:43.908935 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a0898-9994-4500-b004-fbc2152d7dc1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:19:43.909243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a0898-9994-4500-b004-fbc2152d7dc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:43.910662 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a0898-9994-4500-b004-fbc2152d7dc1 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:43.915080 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad88ad-54a3-413b-a281-18f09c2aec06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:43.916609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad88ad-54a3-413b-a281-18f09c2aec06 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:43.923695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:19:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:19:43.929634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a0898-9994-4500-b004-fbc2152d7dc1 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:44.021280 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:19:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 20:19:44.026734 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad88ad-54a3-413b-a281-18f09c2aec06 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:44.029757 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:44.034575 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a0898-9994-4500-b004-fbc2152d7dc1 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:44.068786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459e0c2-c479-4c09-8d7f-e166b95ad389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:44.069662 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459e0c2-c479-4c09-8d7f-e166b95ad389 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:44.069941 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459e0c2-c479-4c09-8d7f-e166b95ad389 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:44.070217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459e0c2-c479-4c09-8d7f-e166b95ad389 None admin] PATH_INFO: `/v3/users/8b5c1280f1184855bd78412238939419` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:44.071473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459e0c2-c479-4c09-8d7f-e166b95ad389 None admin] RBAC: Authorizing `identity:get_user(user_id=8b5c1280f1184855bd78412238939419)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:44.072943 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459e0c2-c479-4c09-8d7f-e166b95ad389 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:44.074597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:19:43 2022] GET /identity/v3/users/8b5c1280f1184855bd78412238939419 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:19:44.080927 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459e0c2-c479-4c09-8d7f-e166b95ad389 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:44.096523 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:44.128981 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad88ad-54a3-413b-a281-18f09c2aec06 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:44.164090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9b259-ef11-4295-8c0d-52820b872c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:44.164893 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9b259-ef11-4295-8c0d-52820b872c24 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:44.165181 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9b259-ef11-4295-8c0d-52820b872c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:44.165450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9b259-ef11-4295-8c0d-52820b872c24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:44.185452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d9b259-ef11-4295-8c0d-52820b872c24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:44.186870 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459e0c2-c479-4c09-8d7f-e166b95ad389 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:44.187027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d9b259-ef11-4295-8c0d-52820b872c24 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:44.208214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-b7d9b259-ef11-4295-8c0d-52820b872c24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:19:44.208214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:19:44.208214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:19:44.208214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:19:44.208214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:19:44.208214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:19:44.208214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:19:44.208214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:19:44.208214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:19:44.208214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:19:44.208214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:19:44.208214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:19:44.208214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:19:44.208214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:19:44.208214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:19:44.209082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:19:44.209082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:19:44.209082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:19:44.209082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:19:44.209082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:19:44.209082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:19:44.209082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:19:44.209082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:19:44.209082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:19:44.209082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:19:44.209082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:19:44.209082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:19:44.209082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:19:44.209082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:19:44.209082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:19:44.209082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:19:44.209082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:19:44.210025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:19:44.210025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:19:44.210025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:19:44.210025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:19:44.210025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 9/18] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:19:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:19:44.215315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9b259-ef11-4295-8c0d-52820b872c24 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:44.222109 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a137b4a-2a88-4c61-a29b-28a55716a0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:44.222899 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a137b4a-2a88-4c61-a29b-28a55716a0b7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:44.223210 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a137b4a-2a88-4c61-a29b-28a55716a0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:44.223483 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a137b4a-2a88-4c61-a29b-28a55716a0b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:44.224667 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a137b4a-2a88-4c61-a29b-28a55716a0b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:44.226099 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a137b4a-2a88-4c61-a29b-28a55716a0b7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:44.227725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 10/19] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:19:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:19:44.233513 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a137b4a-2a88-4c61-a29b-28a55716a0b7 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:44.272899 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:44.318903 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9b259-ef11-4295-8c0d-52820b872c24 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:44.320513 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:44.342032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a137b4a-2a88-4c61-a29b-28a55716a0b7 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:44.354558 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e697dd6-c32e-48b5-aea1-2d24bff8f752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:44.355264 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e697dd6-c32e-48b5-aea1-2d24bff8f752 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:44.355547 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e697dd6-c32e-48b5-aea1-2d24bff8f752 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:44.355825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e697dd6-c32e-48b5-aea1-2d24bff8f752 None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:44.356568 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e697dd6-c32e-48b5-aea1-2d24bff8f752 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:19:44.356877 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e697dd6-c32e-48b5-aea1-2d24bff8f752 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:44.358302 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e697dd6-c32e-48b5-aea1-2d24bff8f752 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:44.375983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b6b27-c50c-4476-95dc-29d63821d0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:44.376478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 10/20] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:19:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:19:44.376643 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b6b27-c50c-4476-95dc-29d63821d0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:44.376926 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b6b27-c50c-4476-95dc-29d63821d0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:44.377199 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b6b27-c50c-4476-95dc-29d63821d0d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:44.378036 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320b6b27-c50c-4476-95dc-29d63821d0d5 None admin] RBAC: Adding query filter params (role.id=5ca112885467422684e134cb0f94865a, scope.domain.id=default, user.id=8b5c1280f1184855bd78412238939419) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:19:44.378344 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320b6b27-c50c-4476-95dc-29d63821d0d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:44.379866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320b6b27-c50c-4476-95dc-29d63821d0d5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:44.388085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 11/21] 10.222.0.52 () {56 vars in 1322 bytes} [Thu Jul 14 20:19:44 2022] GET /identity/v3/role_assignments?user.id=8b5c1280f1184855bd78412238939419&scope.domain.id=default&role.id=5ca112885467422684e134cb0f94865a => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:19:46.840944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04e09f-8999-453a-ad5a-7f54cc07a2d7 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:46.841953 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04e09f-8999-453a-ad5a-7f54cc07a2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:46.842468 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04e09f-8999-453a-ad5a-7f54cc07a2d7 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:46.844622 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 11/22] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:19:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:19:46.850365 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f1226-463d-443b-b2f3-75bdcb56eec3 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:46.851248 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f1226-463d-443b-b2f3-75bdcb56eec3 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:46.851719 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f1226-463d-443b-b2f3-75bdcb56eec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:46.873891 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-fc6f1226-463d-443b-b2f3-75bdcb56eec3 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:19:46.875459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fc6f1226-463d-443b-b2f3-75bdcb56eec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:19:46.880623 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c7834-1848-4e88-975f-023889d55ad2 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:46.881241 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c7834-1848-4e88-975f-023889d55ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:46.881713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c7834-1848-4e88-975f-023889d55ad2 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:46.883365 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 12/23] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:19:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:19:46.896486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fe1f8-8a66-4b77-aa35-4a0ac0ec93ab None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:46.897090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fe1f8-8a66-4b77-aa35-4a0ac0ec93ab None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:46.897556 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fe1f8-8a66-4b77-aa35-4a0ac0ec93ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:46.918394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-085fe1f8-8a66-4b77-aa35-4a0ac0ec93ab None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:19:46.920188 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-085fe1f8-8a66-4b77-aa35-4a0ac0ec93ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:19:47.003932 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 13/24] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:19:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:19:47.005329 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da56ad-8aee-4090-9e61-837eed77a939 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:47.005628 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da56ad-8aee-4090-9e61-837eed77a939 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:47.005911 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da56ad-8aee-4090-9e61-837eed77a939 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:47.007382 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 14/25] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:19:47.013021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39078319-5670-409c-9c6b-4113ac446c9e None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:47.013314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39078319-5670-409c-9c6b-4113ac446c9e None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:47.013598 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39078319-5670-409c-9c6b-4113ac446c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:47.033610 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-39078319-5670-409c-9c6b-4113ac446c9e None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:19:47.034816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-39078319-5670-409c-9c6b-4113ac446c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:19:47.036612 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 12/26] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:19:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:19:47.064378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f1226-463d-443b-b2f3-75bdcb56eec3 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:47.082570 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:47.113805 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 15/27] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:19:47.114674 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39078319-5670-409c-9c6b-4113ac446c9e None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:47.225442 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:47.249064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39078319-5670-409c-9c6b-4113ac446c9e None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:47.272929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f1226-463d-443b-b2f3-75bdcb56eec3 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:47.288719 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25086d0b-9dc9-4e2a-a660-a5efe46d677c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:47.289572 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25086d0b-9dc9-4e2a-a660-a5efe46d677c None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:47.289864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25086d0b-9dc9-4e2a-a660-a5efe46d677c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:47.290143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25086d0b-9dc9-4e2a-a660-a5efe46d677c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:47.309969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25086d0b-9dc9-4e2a-a660-a5efe46d677c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:47.312292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25086d0b-9dc9-4e2a-a660-a5efe46d677c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:47.324308 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471bbdb-93ab-4953-8327-6648e3772c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:47.325123 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471bbdb-93ab-4953-8327-6648e3772c55 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:47.325407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471bbdb-93ab-4953-8327-6648e3772c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:47.325684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471bbdb-93ab-4953-8327-6648e3772c55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:47.339146 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471bbdb-93ab-4953-8327-6648e3772c55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:19:47.347332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-3471bbdb-93ab-4953-8327-6648e3772c55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:19:47.347332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:19:47.347332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:19:47.347332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:19:47.347332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:19:47.347332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:19:47.347332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:19:47.347332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:19:47.347332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:19:47.347332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:19:47.347332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:19:47.347332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:19:47.347332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:19:47.347332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:19:47.347332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:19:47.347332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:19:47.348207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:19:47.348207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:19:47.348207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:19:47.348207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:19:47.348207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:19:47.348207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:19:47.348207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:19:47.348207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:19:47.348207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:19:47.348207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:19:47.348207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:19:47.348207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:19:47.348207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:19:47.348207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:19:47.348207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:19:47.348207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:19:47.349672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:19:47.349672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:19:47.349672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:19:47.349672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:19:47.349672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:19:47.349672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:19:47.349672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:19:47.349672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:19:47.349672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:19:47.349672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:19:47.349672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:19:47.349672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:19:47.349672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 13/28] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:19:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:19:47.349672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471bbdb-93ab-4953-8327-6648e3772c55 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:47.351478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-25086d0b-9dc9-4e2a-a660-a5efe46d677c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:19:47.351478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:19:47.351478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:19:47.351478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:19:47.351478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:19:47.351478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:19:47.351478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:19:47.351478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:19:47.351478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:19:47.351478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:19:47.351478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:19:47.351478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:19:47.351478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:19:47.351478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:19:47.351478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:19:47.352887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:19:47.352887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:19:47.352887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:19:47.352887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:19:47.352887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:19:47.352887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:19:47.352887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:19:47.352887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:19:47.352887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:19:47.352887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:19:47.352887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:19:47.352887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:19:47.352887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:19:47.352887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:19:47.352887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:19:47.352887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:19:47.352887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:19:47.353857 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:19:47.353857 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:19:47.353857 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:19:47.353857 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:19:47.353857 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 16/29] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:19:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:19:47.362506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25086d0b-9dc9-4e2a-a660-a5efe46d677c None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:47.414049 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:47.528661 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471bbdb-93ab-4953-8327-6648e3772c55 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:47.570863 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77bea5-ed35-427b-9ce4-1e8633fbb005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:47.572261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77bea5-ed35-427b-9ce4-1e8633fbb005 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:47.572261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77bea5-ed35-427b-9ce4-1e8633fbb005 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:47.573400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77bea5-ed35-427b-9ce4-1e8633fbb005 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:47.573657 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77bea5-ed35-427b-9ce4-1e8633fbb005 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:47.575155 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77bea5-ed35-427b-9ce4-1e8633fbb005 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:47.582519 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:47.585174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25086d0b-9dc9-4e2a-a660-a5efe46d677c None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:47.647048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74ab23-2e5b-4570-8e2f-5ef9631fe156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:47.647048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74ab23-2e5b-4570-8e2f-5ef9631fe156 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:47.647048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74ab23-2e5b-4570-8e2f-5ef9631fe156 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:47.647048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74ab23-2e5b-4570-8e2f-5ef9631fe156 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:47.647539 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74ab23-2e5b-4570-8e2f-5ef9631fe156 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:19:47.647539 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74ab23-2e5b-4570-8e2f-5ef9631fe156 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:47.647539 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74ab23-2e5b-4570-8e2f-5ef9631fe156 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:47.660189 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 17/30] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:19:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:47.661067 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74ab23-2e5b-4570-8e2f-5ef9631fe156 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:47.743225 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 14/31] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jul 14 20:19:47 2022] POST /identity/v3/roles => generated 219 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:19:47.744034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77bea5-ed35-427b-9ce4-1e8633fbb005 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:47.846096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:47.857521 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74ab23-2e5b-4570-8e2f-5ef9631fe156 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:47.897986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1209905-eca7-4edb-97fe-261d8c926490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:47.898669 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1209905-eca7-4edb-97fe-261d8c926490 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:47.898973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1209905-eca7-4edb-97fe-261d8c926490 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:47.899258 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1209905-eca7-4edb-97fe-261d8c926490 None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:47.900000 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1209905-eca7-4edb-97fe-261d8c926490 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:19:47.900309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1209905-eca7-4edb-97fe-261d8c926490 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:47.901724 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1209905-eca7-4edb-97fe-261d8c926490 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:47.915748 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 18/32] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:47.917221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f087cd-390a-4b67-84ee-b47b607fdf4d None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:47.917523 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f087cd-390a-4b67-84ee-b47b607fdf4d None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:47.917802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f087cd-390a-4b67-84ee-b47b607fdf4d None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:47.919441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 19/33] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:19:47.928308 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f087cd-390a-4b67-84ee-b47b607fdf4d None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:47.942524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77bea5-ed35-427b-9ce4-1e8633fbb005 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:47.990573 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1206d7-ebdf-408c-b740-31c92a44cfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:47.991670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1206d7-ebdf-408c-b740-31c92a44cfef None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:47.991957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1206d7-ebdf-408c-b740-31c92a44cfef None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:47.992238 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1206d7-ebdf-408c-b740-31c92a44cfef None admin] PATH_INFO: `/v3/users/8b5c1280f1184855bd78412238939419` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:47.995884 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1206d7-ebdf-408c-b740-31c92a44cfef None admin] RBAC: Authorizing `identity:get_user(user_id=8b5c1280f1184855bd78412238939419)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:47.998081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1206d7-ebdf-408c-b740-31c92a44cfef None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:48.000800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 15/34] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:19:47 2022] GET /identity/v3/users/8b5c1280f1184855bd78412238939419 => generated 247 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:48.003603 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64f3ec-fd0f-42c8-a641-6a5ca0103b06 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:48.003901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64f3ec-fd0f-42c8-a641-6a5ca0103b06 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:48.004411 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64f3ec-fd0f-42c8-a641-6a5ca0103b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:48.025126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ac64f3ec-fd0f-42c8-a641-6a5ca0103b06 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:19:48.026758 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ac64f3ec-fd0f-42c8-a641-6a5ca0103b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:19:48.095994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f087cd-390a-4b67-84ee-b47b607fdf4d None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:48.102115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:48.119151 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 16/35] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:19:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:19:48.119151 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64f3ec-fd0f-42c8-a641-6a5ca0103b06 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:48.138261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506df34-03b5-4528-b4b0-6d3718f8a7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:48.139043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506df34-03b5-4528-b4b0-6d3718f8a7db None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:48.139330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506df34-03b5-4528-b4b0-6d3718f8a7db None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:48.139639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506df34-03b5-4528-b4b0-6d3718f8a7db None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:48.140665 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506df34-03b5-4528-b4b0-6d3718f8a7db None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:48.142151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506df34-03b5-4528-b4b0-6d3718f8a7db None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:48.234755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64f3ec-fd0f-42c8-a641-6a5ca0103b06 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:48.268489 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0aebe-c165-42fe-8b89-98d1bdfecb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:48.269285 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0aebe-c165-42fe-8b89-98d1bdfecb61 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:48.269570 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0aebe-c165-42fe-8b89-98d1bdfecb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:48.269845 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0aebe-c165-42fe-8b89-98d1bdfecb61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:48.271073 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0aebe-c165-42fe-8b89-98d1bdfecb61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:48.272529 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0aebe-c165-42fe-8b89-98d1bdfecb61 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:48.279005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 17/36] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:19:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:48.279005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611e2e9-0527-46cf-9235-0d4d8a839c57 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:48.279005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611e2e9-0527-46cf-9235-0d4d8a839c57 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:48.279005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611e2e9-0527-46cf-9235-0d4d8a839c57 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:48.279005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 18/37] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:19:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:19:48.279005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611e2e9-0527-46cf-9235-0d4d8a839c57 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:48.287737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 20/38] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 14 20:19:47 2022] POST /identity/v3/projects => generated 301 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:19:48.288771 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506df34-03b5-4528-b4b0-6d3718f8a7db None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:48.291949 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:48.357990 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:48.413529 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506df34-03b5-4528-b4b0-6d3718f8a7db None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:48.417202 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611e2e9-0527-46cf-9235-0d4d8a839c57 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:48.454364 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223343ef-1867-48f4-b8e9-ef9b5a110c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:48.455383 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223343ef-1867-48f4-b8e9-ef9b5a110c3c None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:48.455674 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223343ef-1867-48f4-b8e9-ef9b5a110c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:48.456567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223343ef-1867-48f4-b8e9-ef9b5a110c3c None admin] PATH_INFO: `/v3/domains/default/users/8b5c1280f1184855bd78412238939419/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:48.458793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223343ef-1867-48f4-b8e9-ef9b5a110c3c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8b5c1280f1184855bd78412238939419, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:48.460465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223343ef-1867-48f4-b8e9-ef9b5a110c3c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:48.481612 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 21/39] 10.222.0.52 () {58 vars in 1361 bytes} [Thu Jul 14 20:19:48 2022] PUT /identity/v3/domains/default/users/8b5c1280f1184855bd78412238939419/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:19:48.482807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61587e4-d5ee-4b16-bf92-295f513e657a None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:48.483162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61587e4-d5ee-4b16-bf92-295f513e657a None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:48.483460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61587e4-d5ee-4b16-bf92-295f513e657a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:48.504038 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-e61587e4-d5ee-4b16-bf92-295f513e657a None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:19:48.505847 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e61587e4-d5ee-4b16-bf92-295f513e657a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:19:48.572403 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7b2ca-d2a6-4f39-84e8-bfb1b74db14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:48.573106 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7b2ca-d2a6-4f39-84e8-bfb1b74db14b None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:48.573397 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7b2ca-d2a6-4f39-84e8-bfb1b74db14b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:48.573677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7b2ca-d2a6-4f39-84e8-bfb1b74db14b None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:48.574559 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7b2ca-d2a6-4f39-84e8-bfb1b74db14b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:48.576032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7b2ca-d2a6-4f39-84e8-bfb1b74db14b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:48.580631 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:48.594522 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 19/40] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jul 14 20:19:48 2022] POST /identity/v3/roles => generated 225 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:19:48.596201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 22/41] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:19:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:19:48.596795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0a76c-cdf2-4964-ae26-d4740db4feb7 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:48.597095 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0a76c-cdf2-4964-ae26-d4740db4feb7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:48.597438 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0a76c-cdf2-4964-ae26-d4740db4feb7 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:48.599032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 20/42] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:19:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:19:48.604615 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f4a0f-b79f-47a6-b73a-1bfc1c89909a None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:48.605058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f4a0f-b79f-47a6-b73a-1bfc1c89909a None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:48.605351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f4a0f-b79f-47a6-b73a-1bfc1c89909a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:48.623023 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0a76c-cdf2-4964-ae26-d4740db4feb7 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:48.635264 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-000f4a0f-b79f-47a6-b73a-1bfc1c89909a None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:19:48.636750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-000f4a0f-b79f-47a6-b73a-1bfc1c89909a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:19:48.741161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 23/43] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:19:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:19:48.767346 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f4a0f-b79f-47a6-b73a-1bfc1c89909a None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:48.780813 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0a76c-cdf2-4964-ae26-d4740db4feb7 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:48.808516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:48.829479 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b699a-a465-4cc5-8d82-c935b99c32c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:48.830293 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b699a-a465-4cc5-8d82-c935b99c32c5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:48.830748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b699a-a465-4cc5-8d82-c935b99c32c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:48.833604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b699a-a465-4cc5-8d82-c935b99c32c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:48.834915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b699a-a465-4cc5-8d82-c935b99c32c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:48.836508 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b699a-a465-4cc5-8d82-c935b99c32c5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:48.838246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 21/44] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:19:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:19:48.844063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b699a-a465-4cc5-8d82-c935b99c32c5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:48.893717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f4a0f-b79f-47a6-b73a-1bfc1c89909a None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:48.921227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:48.933982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0784e1-9255-4614-aa7d-ee3b258c7533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:48.934668 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0784e1-9255-4614-aa7d-ee3b258c7533 None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:48.934981 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0784e1-9255-4614-aa7d-ee3b258c7533 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:48.935295 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0784e1-9255-4614-aa7d-ee3b258c7533 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:48.936040 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0784e1-9255-4614-aa7d-ee3b258c7533 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:48.937977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0784e1-9255-4614-aa7d-ee3b258c7533 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:48.954142 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 24/45] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jul 14 20:19:48 2022] POST /identity/v3/roles => generated 223 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:19:48.967160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b699a-a465-4cc5-8d82-c935b99c32c5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:49.009175 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911dcaae-29f4-4df8-a793-18759493be26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:49.009869 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911dcaae-29f4-4df8-a793-18759493be26 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:49.010157 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911dcaae-29f4-4df8-a793-18759493be26 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:49.010430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911dcaae-29f4-4df8-a793-18759493be26 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:49.013502 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911dcaae-29f4-4df8-a793-18759493be26 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:49.013502 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911dcaae-29f4-4df8-a793-18759493be26 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:49.050975 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 22/46] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 14 20:19:48 2022] POST /identity/v3/projects => generated 312 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:19:50.577106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d65c5-70b2-4290-a576-d52beb24530a None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:50.577647 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d65c5-70b2-4290-a576-d52beb24530a None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:50.577647 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d65c5-70b2-4290-a576-d52beb24530a None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:50.579062 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 25/47] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:19:50.584351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9520df9-cb12-4ce2-946a-e2634d1fe9ef None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:50.584737 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9520df9-cb12-4ce2-946a-e2634d1fe9ef None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:50.585040 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9520df9-cb12-4ce2-946a-e2634d1fe9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:50.602372 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e9520df9-cb12-4ce2-946a-e2634d1fe9ef None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:19:50.603629 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e9520df9-cb12-4ce2-946a-e2634d1fe9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:19:50.641584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:50.683806 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 23/48] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:19:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:19:50.711215 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d65c5-70b2-4290-a576-d52beb24530a None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:50.815826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d65c5-70b2-4290-a576-d52beb24530a None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:50.852277 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fb740f-e5ea-4bdc-8180-27c1ee979f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:50.853002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb740f-e5ea-4bdc-8180-27c1ee979f38 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:50.853291 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb740f-e5ea-4bdc-8180-27c1ee979f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:50.853572 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb740f-e5ea-4bdc-8180-27c1ee979f38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:50.865678 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fb740f-e5ea-4bdc-8180-27c1ee979f38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:19:50.874923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-d7fb740f-e5ea-4bdc-8180-27c1ee979f38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:19:50.874923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:19:50.874923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:19:50.874923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:19:50.874923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:19:50.874923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:19:50.874923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:19:50.874923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:19:50.874923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:19:50.874923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:19:50.874923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:19:50.874923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:19:50.874923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:19:50.874923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:19:50.874923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:19:50.874923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:19:50.875806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:19:50.875806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:19:50.875806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:19:50.875806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:19:50.875806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:19:50.875806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:19:50.875806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:19:50.875806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:19:50.875806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:19:50.875806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:19:50.875806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:19:50.875806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:19:50.875806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:19:50.875806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:19:50.875806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:19:50.875806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:19:50.876706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:19:50.876706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:19:50.876706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:19:50.876706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:19:50.876706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:19:50.876706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:19:50.876706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:19:50.876706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:19:50.876706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:19:50.876706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:19:50.876706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:19:50.876706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:19:50.876706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 26/49] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:19:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:19:50.882027 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9520df9-cb12-4ce2-946a-e2634d1fe9ef None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:50.981652 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9520df9-cb12-4ce2-946a-e2634d1fe9ef None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:51.016020 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520558ce-647e-448c-b36c-d77f0da2bf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:51.016683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520558ce-647e-448c-b36c-d77f0da2bf38 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:51.016966 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520558ce-647e-448c-b36c-d77f0da2bf38 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:51.017238 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520558ce-647e-448c-b36c-d77f0da2bf38 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:51.018241 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520558ce-647e-448c-b36c-d77f0da2bf38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:19:51.018541 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520558ce-647e-448c-b36c-d77f0da2bf38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:51.019957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520558ce-647e-448c-b36c-d77f0da2bf38 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:51.028280 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 24/50] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:19:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:51.034514 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fb740f-e5ea-4bdc-8180-27c1ee979f38 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:51.059996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed14f0-c1ac-43df-92e3-04301fd9493f None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:51.060298 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed14f0-c1ac-43df-92e3-04301fd9493f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:51.060678 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed14f0-c1ac-43df-92e3-04301fd9493f None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:51.062491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 25/51] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:19:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:19:51.067821 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee13f09-c84c-4ea3-8ea3-fed68f0e9a0d None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:51.068112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee13f09-c84c-4ea3-8ea3-fed68f0e9a0d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:51.068402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee13f09-c84c-4ea3-8ea3-fed68f0e9a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:51.080754 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:51.085474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-7ee13f09-c84c-4ea3-8ea3-fed68f0e9a0d None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:19:51.086706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7ee13f09-c84c-4ea3-8ea3-fed68f0e9a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:19:51.119133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:51.140919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fb740f-e5ea-4bdc-8180-27c1ee979f38 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:51.162259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 26/52] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:19:51.174054 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ef540-d954-4674-9020-2a997c1e7875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:51.174930 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ef540-d954-4674-9020-2a997c1e7875 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:51.175243 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ef540-d954-4674-9020-2a997c1e7875 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:51.175520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ef540-d954-4674-9020-2a997c1e7875 None admin] PATH_INFO: `/v3/users/8b5c1280f1184855bd78412238939419` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:51.176709 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ef540-d954-4674-9020-2a997c1e7875 None admin] RBAC: Authorizing `identity:get_user(user_id=8b5c1280f1184855bd78412238939419)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:51.178162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ef540-d954-4674-9020-2a997c1e7875 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:51.179814 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 27/53] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:19:51 2022] GET /identity/v3/users/8b5c1280f1184855bd78412238939419 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:19:51.184925 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee13f09-c84c-4ea3-8ea3-fed68f0e9a0d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:51.185132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ef540-d954-4674-9020-2a997c1e7875 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:51.296532 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee13f09-c84c-4ea3-8ea3-fed68f0e9a0d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:51.333702 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5bb7c-b33e-4839-8acf-ba3d662705e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:51.334509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5bb7c-b33e-4839-8acf-ba3d662705e0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:51.334795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5bb7c-b33e-4839-8acf-ba3d662705e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:51.335107 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5bb7c-b33e-4839-8acf-ba3d662705e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:51.336300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac5bb7c-b33e-4839-8acf-ba3d662705e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:51.337746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac5bb7c-b33e-4839-8acf-ba3d662705e0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:51.339569 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 27/54] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:19:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:51.345061 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5bb7c-b33e-4839-8acf-ba3d662705e0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:51.377085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:51.394642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ef540-d954-4674-9020-2a997c1e7875 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:51.405436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:51.429415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd5d29-539c-4d57-90b7-de2ae1938f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:51.430212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd5d29-539c-4d57-90b7-de2ae1938f26 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:51.430494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd5d29-539c-4d57-90b7-de2ae1938f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:51.430765 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd5d29-539c-4d57-90b7-de2ae1938f26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:51.431976 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd5d29-539c-4d57-90b7-de2ae1938f26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:51.433418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd5d29-539c-4d57-90b7-de2ae1938f26 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:51.435054 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 28/55] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:19:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:19:51.441429 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd5d29-539c-4d57-90b7-de2ae1938f26 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:51.449189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5bb7c-b33e-4839-8acf-ba3d662705e0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:51.491106 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3482e-b9f2-4262-91e2-d83e8bc2e560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:51.491773 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3482e-b9f2-4262-91e2-d83e8bc2e560 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:51.492060 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3482e-b9f2-4262-91e2-d83e8bc2e560 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:51.492334 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3482e-b9f2-4262-91e2-d83e8bc2e560 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:51.493049 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3482e-b9f2-4262-91e2-d83e8bc2e560 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:51.494503 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3482e-b9f2-4262-91e2-d83e8bc2e560 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:51.523668 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 28/56] 10.222.0.52 () {60 vars in 1140 bytes} [Thu Jul 14 20:19:51 2022] POST /identity/v3/projects => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:19:51.556595 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd5d29-539c-4d57-90b7-de2ae1938f26 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:51.595598 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d1ee4-28d6-4aa8-a5f5-95b8f33d74e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:51.596384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d1ee4-28d6-4aa8-a5f5-95b8f33d74e0 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:51.596737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d1ee4-28d6-4aa8-a5f5-95b8f33d74e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:51.597082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d1ee4-28d6-4aa8-a5f5-95b8f33d74e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:51.598009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d1ee4-28d6-4aa8-a5f5-95b8f33d74e0 None admin] RBAC: Adding query filter params (role.id=5ca112885467422684e134cb0f94865a, scope.domain.id=default, user.id=8b5c1280f1184855bd78412238939419) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:19:51.598391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d1ee4-28d6-4aa8-a5f5-95b8f33d74e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:51.599965 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d1ee4-28d6-4aa8-a5f5-95b8f33d74e0 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:51.609778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 29/57] 10.222.0.52 () {56 vars in 1322 bytes} [Thu Jul 14 20:19:51 2022] GET /identity/v3/role_assignments?user.id=8b5c1280f1184855bd78412238939419&scope.domain.id=default&role.id=5ca112885467422684e134cb0f94865a => generated 530 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:19:53.568193 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998adf08-9061-43d5-bb3f-b448c0e04a33 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:53.568714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998adf08-9061-43d5-bb3f-b448c0e04a33 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:53.568797 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998adf08-9061-43d5-bb3f-b448c0e04a33 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:53.570138 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 29/58] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:19:53.575713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fac023-d487-4072-8f2e-9606735ed5c7 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:53.576124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fac023-d487-4072-8f2e-9606735ed5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:53.576274 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fac023-d487-4072-8f2e-9606735ed5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:53.593433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-d8fac023-d487-4072-8f2e-9606735ed5c7 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:19:53.594636 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d8fac023-d487-4072-8f2e-9606735ed5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:19:53.630218 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:53.663672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 30/59] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:19:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:19:53.688141 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998adf08-9061-43d5-bb3f-b448c0e04a33 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:53.769869 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:53.789528 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998adf08-9061-43d5-bb3f-b448c0e04a33 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:53.823442 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b1feb-91b2-4f32-9304-22c5df9985ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:53.824160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b1feb-91b2-4f32-9304-22c5df9985ca None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:53.824446 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b1feb-91b2-4f32-9304-22c5df9985ca None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:53.824723 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b1feb-91b2-4f32-9304-22c5df9985ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:53.825904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b1feb-91b2-4f32-9304-22c5df9985ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:53.827436 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b1feb-91b2-4f32-9304-22c5df9985ca None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:53.829073 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 30/60] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:19:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:53.834738 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fac023-d487-4072-8f2e-9606735ed5c7 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:53.934372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fac023-d487-4072-8f2e-9606735ed5c7 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:53.968193 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539ffe6-ce40-4c34-aa01-f3ae3834d25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:53.968869 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539ffe6-ce40-4c34-aa01-f3ae3834d25e None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:53.969156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539ffe6-ce40-4c34-aa01-f3ae3834d25e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:53.969439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539ffe6-ce40-4c34-aa01-f3ae3834d25e None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:53.970157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539ffe6-ce40-4c34-aa01-f3ae3834d25e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:53.971634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539ffe6-ce40-4c34-aa01-f3ae3834d25e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:53.998087 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 31/61] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 14 20:19:53 2022] POST /identity/v3/users => generated 275 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:19:57.582373 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f323d-8ef5-47cc-8301-05b50ab576b9 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:57.582957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f323d-8ef5-47cc-8301-05b50ab576b9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:57.582957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f323d-8ef5-47cc-8301-05b50ab576b9 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:57.584394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 31/62] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:19:57.590387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af3d01-994d-4ce8-80e3-c20d03bd0a5e None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:57.590889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af3d01-994d-4ce8-80e3-c20d03bd0a5e None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:57.591154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af3d01-994d-4ce8-80e3-c20d03bd0a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:57.614561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-b7af3d01-994d-4ce8-80e3-c20d03bd0a5e None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:19:57.615723 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651df4f-761f-4d31-a652-b1e5d1e58512 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:57.615848 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b7af3d01-994d-4ce8-80e3-c20d03bd0a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:19:57.616330 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651df4f-761f-4d31-a652-b1e5d1e58512 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:57.616824 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651df4f-761f-4d31-a652-b1e5d1e58512 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:57.618447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 32/63] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:19:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:19:57.624393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13d6ac-996d-48ab-96e5-3a02c1e6baf7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:57.625026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13d6ac-996d-48ab-96e5-3a02c1e6baf7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:57.625502 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13d6ac-996d-48ab-96e5-3a02c1e6baf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:57.652829 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-eb13d6ac-996d-48ab-96e5-3a02c1e6baf7 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:19:57.654416 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-eb13d6ac-996d-48ab-96e5-3a02c1e6baf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:19:57.667337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:57.723679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 32/64] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:19:57.744669 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff026e-6da7-4d07-9b3d-4a98218bebfc None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:57.745354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff026e-6da7-4d07-9b3d-4a98218bebfc None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:57.745797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff026e-6da7-4d07-9b3d-4a98218bebfc None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:57.747676 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 33/65] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:19:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:19:57.749576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff026e-6da7-4d07-9b3d-4a98218bebfc None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:57.757589 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 33/66] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:19:57.763318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a65a19-195c-4741-9978-33cec2001d06 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:57.763808 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a65a19-195c-4741-9978-33cec2001d06 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:57.764233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a65a19-195c-4741-9978-33cec2001d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:57.783389 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-87a65a19-195c-4741-9978-33cec2001d06 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:19:57.784927 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-87a65a19-195c-4741-9978-33cec2001d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:19:57.869264 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 34/67] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:19:57.870335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a65a19-195c-4741-9978-33cec2001d06 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:57.888746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:57.902084 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff026e-6da7-4d07-9b3d-4a98218bebfc None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:57.940599 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd19d2-c275-4825-bb25-4d0abe7aebac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:57.941392 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd19d2-c275-4825-bb25-4d0abe7aebac None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:57.941832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd19d2-c275-4825-bb25-4d0abe7aebac None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:57.942245 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd19d2-c275-4825-bb25-4d0abe7aebac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:57.953760 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd19d2-c275-4825-bb25-4d0abe7aebac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:19:57.961634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-97fd19d2-c275-4825-bb25-4d0abe7aebac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:19:57.961634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:19:57.961634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:19:57.961634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:19:57.961634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:19:57.961634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:19:57.961634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:19:57.961634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:19:57.961634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:19:57.961634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:19:57.961634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:19:57.961634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:19:57.961634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:19:57.961634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:19:57.961634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:19:57.961634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:19:57.962715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:19:57.962715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:19:57.962715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:19:57.962715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:19:57.962715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:19:57.962715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:19:57.962715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:19:57.962715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:19:57.962715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:19:57.962715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:19:57.962715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:19:57.962715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:19:57.962715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:19:57.962715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:19:57.962715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:19:57.962715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:19:57.963574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:19:57.963574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:19:57.963574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:19:57.963574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:19:57.963574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:19:57.963574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:19:57.963574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:19:57.963574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:19:57.963574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:19:57.963574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:19:57.963574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:19:57.963574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:19:57.965861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 34/68] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:19:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:19:57.967299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378ecd8-c9e1-45be-b510-0b8a2581517b None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:57.967652 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378ecd8-c9e1-45be-b510-0b8a2581517b None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:57.967931 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378ecd8-c9e1-45be-b510-0b8a2581517b None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:57.969210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 35/69] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:19:57.970019 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378ecd8-c9e1-45be-b510-0b8a2581517b None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:57.991084 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:57.996443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a65a19-195c-4741-9978-33cec2001d06 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:58.033096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81856e-4903-46b6-9f06-fb44e7fd4c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:58.033900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81856e-4903-46b6-9f06-fb44e7fd4c83 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:58.034350 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81856e-4903-46b6-9f06-fb44e7fd4c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:58.034636 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81856e-4903-46b6-9f06-fb44e7fd4c83 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:58.045760 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81856e-4903-46b6-9f06-fb44e7fd4c83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:19:58.053487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-db81856e-4903-46b6-9f06-fb44e7fd4c83 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:19:58.053487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:19:58.053487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:19:58.053487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:19:58.053487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:19:58.053487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:19:58.053487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:19:58.053487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:19:58.053487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:19:58.053487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:19:58.053487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:19:58.053487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:19:58.053487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:19:58.053487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:19:58.053487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:19:58.053487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:19:58.054346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:19:58.054346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:19:58.054346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:19:58.054346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:19:58.054346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:19:58.054346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:19:58.054346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:19:58.054346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:19:58.054346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:19:58.054346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:19:58.054346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:19:58.054346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:19:58.054346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:19:58.054346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:19:58.054346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:19:58.054346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:19:58.055145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:19:58.055145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:19:58.055145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:19:58.055145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:19:58.055145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:19:58.055145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:19:58.055145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:19:58.055145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:19:58.055145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:19:58.055145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:19:58.055145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:19:58.055145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:19:58.055145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 35/70] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:19:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:19:58.058564 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81856e-4903-46b6-9f06-fb44e7fd4c83 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:58.085756 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378ecd8-c9e1-45be-b510-0b8a2581517b None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:58.121510 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c63df-a711-4ef2-b431-db4687018883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:58.122243 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c63df-a711-4ef2-b431-db4687018883 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:58.122528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c63df-a711-4ef2-b431-db4687018883 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:58.122806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c63df-a711-4ef2-b431-db4687018883 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:58.135879 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c63df-a711-4ef2-b431-db4687018883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:19:58.143885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:58.144970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-7c1c63df-a711-4ef2-b431-db4687018883 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:19:58.144970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:19:58.144970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:19:58.144970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:19:58.144970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:19:58.144970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:19:58.144970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:19:58.144970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:19:58.144970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:19:58.144970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:19:58.144970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:19:58.144970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:19:58.144970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:19:58.144970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:19:58.144970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:19:58.144970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:19:58.145850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:19:58.145850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:19:58.145850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:19:58.145850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:19:58.145850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:19:58.145850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:19:58.145850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:19:58.145850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:19:58.145850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:19:58.145850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:19:58.145850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:19:58.145850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:19:58.145850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:19:58.145850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:19:58.145850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:19:58.145850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:19:58.146592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:19:58.146592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:19:58.146592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:19:58.146592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:19:58.146592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:19:58.146592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:19:58.146592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:19:58.146592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:19:58.146592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:19:58.146592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:19:58.146592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:19:58.146592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:19:58.146592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 36/71] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:19:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:19:58.152571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4f46a-60a5-46c7-9efc-7b67d4f0dfca None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:58.152871 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4f46a-60a5-46c7-9efc-7b67d4f0dfca None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:58.153164 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4f46a-60a5-46c7-9efc-7b67d4f0dfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:58.171108 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-9ae4f46a-60a5-46c7-9efc-7b67d4f0dfca None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:19:58.172334 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9ae4f46a-60a5-46c7-9efc-7b67d4f0dfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:19:58.179889 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81856e-4903-46b6-9f06-fb44e7fd4c83 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:58.215862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eec967-7510-40d0-9297-9c56c443c106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:58.216539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eec967-7510-40d0-9297-9c56c443c106 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:58.216827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eec967-7510-40d0-9297-9c56c443c106 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:58.217104 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eec967-7510-40d0-9297-9c56c443c106 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:58.218204 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eec967-7510-40d0-9297-9c56c443c106 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:19:58.218519 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eec967-7510-40d0-9297-9c56c443c106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:58.219999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eec967-7510-40d0-9297-9c56c443c106 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:58.229729 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 36/72] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:19:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:58.230466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eec967-7510-40d0-9297-9c56c443c106 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:58.244314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 37/73] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:19:58.245370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4f46a-60a5-46c7-9efc-7b67d4f0dfca None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:58.257774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:58.337255 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eec967-7510-40d0-9297-9c56c443c106 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:58.349505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4f46a-60a5-46c7-9efc-7b67d4f0dfca None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:58.366624 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:58.372705 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af011c9-8f6e-49c0-ba29-090e6a1eee89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:58.373322 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af011c9-8f6e-49c0-ba29-090e6a1eee89 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:58.373608 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af011c9-8f6e-49c0-ba29-090e6a1eee89 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:58.374522 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af011c9-8f6e-49c0-ba29-090e6a1eee89 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:58.375329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af011c9-8f6e-49c0-ba29-090e6a1eee89 None admin] RBAC: Adding query filter params (name=member) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:19:58.375643 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af011c9-8f6e-49c0-ba29-090e6a1eee89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:58.377027 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af011c9-8f6e-49c0-ba29-090e6a1eee89 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:58.384773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c0fec-2876-4838-90df-8b6c825557a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:58.385345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 37/74] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:19:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:58.385436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c0fec-2876-4838-90df-8b6c825557a3 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:58.385729 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c0fec-2876-4838-90df-8b6c825557a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:58.386015 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c0fec-2876-4838-90df-8b6c825557a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:58.386611 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aced44-da23-44b6-a509-59fea4b4386d None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:58.386790 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c0fec-2876-4838-90df-8b6c825557a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:19:58.386905 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aced44-da23-44b6-a509-59fea4b4386d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:58.387134 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c0fec-2876-4838-90df-8b6c825557a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:58.387218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aced44-da23-44b6-a509-59fea4b4386d None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:58.390979 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 38/75] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:19:58.391104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c0fec-2876-4838-90df-8b6c825557a3 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:58.391197 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aced44-da23-44b6-a509-59fea4b4386d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:58.397915 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 38/76] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:19:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:58.398750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c0fec-2876-4838-90df-8b6c825557a3 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:58.472376 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:58.499937 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aced44-da23-44b6-a509-59fea4b4386d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:58.505126 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c0fec-2876-4838-90df-8b6c825557a3 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:58.534650 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191548fc-5a33-4063-b0df-1be18b9c627e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:58.535555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191548fc-5a33-4063-b0df-1be18b9c627e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:58.535846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191548fc-5a33-4063-b0df-1be18b9c627e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:58.536127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191548fc-5a33-4063-b0df-1be18b9c627e None admin] PATH_INFO: `/v3/users/8b5c1280f1184855bd78412238939419` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:58.537341 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191548fc-5a33-4063-b0df-1be18b9c627e None admin] RBAC: Authorizing `identity:get_user(user_id=8b5c1280f1184855bd78412238939419)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:58.538522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f99cb5-eb00-4506-8c79-5b53fe837d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:58.538808 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191548fc-5a33-4063-b0df-1be18b9c627e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:58.539229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f99cb5-eb00-4506-8c79-5b53fe837d77 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:58.539508 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f99cb5-eb00-4506-8c79-5b53fe837d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:58.539783 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f99cb5-eb00-4506-8c79-5b53fe837d77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:58.540507 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 39/77] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:19:58 2022] GET /identity/v3/users/8b5c1280f1184855bd78412238939419 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:58.540984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f99cb5-eb00-4506-8c79-5b53fe837d77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:58.541276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191548fc-5a33-4063-b0df-1be18b9c627e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:58.542553 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f99cb5-eb00-4506-8c79-5b53fe837d77 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:58.544246 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 39/78] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:19:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:58.545439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a73014-6afd-47b5-9fec-02c96866d0a2 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:58.546649 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a73014-6afd-47b5-9fec-02c96866d0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:58.546957 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a73014-6afd-47b5-9fec-02c96866d0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:58.564752 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-a7a73014-6afd-47b5-9fec-02c96866d0a2 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:19:58.565985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-a7a73014-6afd-47b5-9fec-02c96866d0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:19:58.605764 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:58.634330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 40/79] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:19:58.635032 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a73014-6afd-47b5-9fec-02c96866d0a2 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:58.650838 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191548fc-5a33-4063-b0df-1be18b9c627e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:58.686786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2659fa-55a7-4749-a3c3-9caed1f72e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:58.687681 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2659fa-55a7-4749-a3c3-9caed1f72e1b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:58.687963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2659fa-55a7-4749-a3c3-9caed1f72e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:58.688235 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2659fa-55a7-4749-a3c3-9caed1f72e1b None admin] PATH_INFO: `/v3/users/2f2bd63cb926471a9ba1d25571f4dd51` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:58.702135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:58.702972 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2659fa-55a7-4749-a3c3-9caed1f72e1b None admin] RBAC: Authorizing `identity:get_user(user_id=2f2bd63cb926471a9ba1d25571f4dd51)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:58.704470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2659fa-55a7-4749-a3c3-9caed1f72e1b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:58.706209 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 40/80] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:19:58 2022] GET /identity/v3/users/2f2bd63cb926471a9ba1d25571f4dd51 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:58.707017 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2659fa-55a7-4749-a3c3-9caed1f72e1b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:58.736819 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a73014-6afd-47b5-9fec-02c96866d0a2 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:58.769912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe28abb-19ef-45b0-baa1-9453debf8464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:58.770891 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe28abb-19ef-45b0-baa1-9453debf8464 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:58.771276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe28abb-19ef-45b0-baa1-9453debf8464 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:58.771636 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe28abb-19ef-45b0-baa1-9453debf8464 None admin] PATH_INFO: `/v3/users/2f2bd63cb926471a9ba1d25571f4dd51` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:58.772889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe28abb-19ef-45b0-baa1-9453debf8464 None admin] RBAC: Authorizing `identity:get_user(user_id=2f2bd63cb926471a9ba1d25571f4dd51)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:58.774425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe28abb-19ef-45b0-baa1-9453debf8464 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:58.776082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 41/81] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:19:58 2022] GET /identity/v3/users/2f2bd63cb926471a9ba1d25571f4dd51 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:58.776813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe28abb-19ef-45b0-baa1-9453debf8464 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:58.817596 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2659fa-55a7-4749-a3c3-9caed1f72e1b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:58.838441 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:58.852701 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009a4a5-5c98-4c3f-a329-da60eb7dbc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:58.853347 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009a4a5-5c98-4c3f-a329-da60eb7dbc8d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:58.853631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009a4a5-5c98-4c3f-a329-da60eb7dbc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:58.853903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009a4a5-5c98-4c3f-a329-da60eb7dbc8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:58.887086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe28abb-19ef-45b0-baa1-9453debf8464 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:58.922989 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009a4a5-5c98-4c3f-a329-da60eb7dbc8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:58.924051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239b569-7664-4409-a019-0faf23c9cf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:58.924784 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009a4a5-5c98-4c3f-a329-da60eb7dbc8d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:58.925008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239b569-7664-4409-a019-0faf23c9cf20 None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:58.925306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239b569-7664-4409-a019-0faf23c9cf20 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:58.925587 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239b569-7664-4409-a019-0faf23c9cf20 None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:58.926314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239b569-7664-4409-a019-0faf23c9cf20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:58.927785 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239b569-7664-4409-a019-0faf23c9cf20 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:58.935704 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:58.949042 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 42/82] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 14 20:19:58 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:19:58.950061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239b569-7664-4409-a019-0faf23c9cf20 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:59.043959 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 41/83] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:19:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:19:59.043959 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009a4a5-5c98-4c3f-a329-da60eb7dbc8d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:59.064479 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:59.075245 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239b569-7664-4409-a019-0faf23c9cf20 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:59.119418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244724e-6677-4b4b-8df1-1b43513bf0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:59.120239 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244724e-6677-4b4b-8df1-1b43513bf0ab None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:59.120530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244724e-6677-4b4b-8df1-1b43513bf0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:59.120832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244724e-6677-4b4b-8df1-1b43513bf0ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:59.132682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244724e-6677-4b4b-8df1-1b43513bf0ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:19:59.140579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-0244724e-6677-4b4b-8df1-1b43513bf0ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:19:59.140579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:19:59.140579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:19:59.140579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:19:59.140579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:19:59.140579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:19:59.140579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:19:59.140579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:19:59.140579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:19:59.140579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:19:59.140579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:19:59.140579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:19:59.140579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:19:59.140579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:19:59.140579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:19:59.140579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:19:59.141416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:19:59.141416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:19:59.141416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:19:59.141416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:19:59.141416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:19:59.141416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:19:59.141416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:19:59.141416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:19:59.141416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:19:59.141416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:19:59.141416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:19:59.141416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:19:59.141416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:19:59.141416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:19:59.141416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:19:59.141416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:19:59.142212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:19:59.142212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:19:59.142212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:19:59.142212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:19:59.142212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:19:59.142212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:19:59.142212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:19:59.142212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:19:59.142212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:19:59.142212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:19:59.142212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:19:59.142212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:19:59.142212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 43/84] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:19:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:19:59.144016 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244724e-6677-4b4b-8df1-1b43513bf0ab None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:59.158172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009a4a5-5c98-4c3f-a329-da60eb7dbc8d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:59.201515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2087df-acea-4ab4-a33d-fb253506feb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:59.202224 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2087df-acea-4ab4-a33d-fb253506feb5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:59.202518 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2087df-acea-4ab4-a33d-fb253506feb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:59.202812 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2087df-acea-4ab4-a33d-fb253506feb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:59.206862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:59.276824 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244724e-6677-4b4b-8df1-1b43513bf0ab None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:59.294692 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2087df-acea-4ab4-a33d-fb253506feb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:59.296815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2087df-acea-4ab4-a33d-fb253506feb5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:59.323760 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:59.323939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf631f-3a26-43ba-a8e7-0f9e45ecc6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:59.324930 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf631f-3a26-43ba-a8e7-0f9e45ecc6ed None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:59.325219 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf631f-3a26-43ba-a8e7-0f9e45ecc6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:59.325702 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf631f-3a26-43ba-a8e7-0f9e45ecc6ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:59.412829 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf631f-3a26-43ba-a8e7-0f9e45ecc6ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:59.415043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf631f-3a26-43ba-a8e7-0f9e45ecc6ed None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:59.430258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 42/85] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:19:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:19:59.431255 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2087df-acea-4ab4-a33d-fb253506feb5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:59.489115 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:59.557721 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 44/86] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:19:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:19:59.558796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf631f-3a26-43ba-a8e7-0f9e45ecc6ed None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:59.573504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2087df-acea-4ab4-a33d-fb253506feb5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:59.599684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:59.617486 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997078d5-bacd-4004-ad8b-4fa92f774fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:59.618356 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997078d5-bacd-4004-ad8b-4fa92f774fee None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:59.618640 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997078d5-bacd-4004-ad8b-4fa92f774fee None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:59.619040 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997078d5-bacd-4004-ad8b-4fa92f774fee None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:59.620365 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997078d5-bacd-4004-ad8b-4fa92f774fee None admin] RBAC: Authorizing `identity:get_project(project_id=930b378be44b4bb7a2009c72a424f1d6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:59.621838 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997078d5-bacd-4004-ad8b-4fa92f774fee None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:59.624078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 43/87] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:19:59 2022] GET /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:59.624939 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997078d5-bacd-4004-ad8b-4fa92f774fee None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:59.697944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf631f-3a26-43ba-a8e7-0f9e45ecc6ed None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:59.743882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b198f-46fd-48d8-8ec0-f16afd864418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:59.744589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b198f-46fd-48d8-8ec0-f16afd864418 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:59.744872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b198f-46fd-48d8-8ec0-f16afd864418 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:59.745148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b198f-46fd-48d8-8ec0-f16afd864418 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:59.745582 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997078d5-bacd-4004-ad8b-4fa92f774fee None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:59.745907 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b198f-46fd-48d8-8ec0-f16afd864418 None admin] RBAC: Adding query filter params (name=member) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:19:59.746213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b198f-46fd-48d8-8ec0-f16afd864418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:59.747686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b198f-46fd-48d8-8ec0-f16afd864418 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:59.756550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 45/88] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:19:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:59.757398 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b198f-46fd-48d8-8ec0-f16afd864418 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:59.779512 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:59.795200 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c94ce-510d-4b24-a15b-673bcfcbc6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:59.796647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c94ce-510d-4b24-a15b-673bcfcbc6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:59.797107 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c94ce-510d-4b24-a15b-673bcfcbc6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:59.797833 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c94ce-510d-4b24-a15b-673bcfcbc6b5 None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:59.799998 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c94ce-510d-4b24-a15b-673bcfcbc6b5 None admin] RBAC: Authorizing `identity:get_project(project_id=930b378be44b4bb7a2009c72a424f1d6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:59.801985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c94ce-510d-4b24-a15b-673bcfcbc6b5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:59.809537 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 44/89] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:19:59 2022] GET /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:59.809537 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c94ce-510d-4b24-a15b-673bcfcbc6b5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:59.872428 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:19:59.894159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b198f-46fd-48d8-8ec0-f16afd864418 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:59.920068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c94ce-510d-4b24-a15b-673bcfcbc6b5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:19:59.933563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b1f10-59bb-4475-874a-8551508b3edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:59.934825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b1f10-59bb-4475-874a-8551508b3edf None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:59.935442 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b1f10-59bb-4475-874a-8551508b3edf None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:59.935918 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b1f10-59bb-4475-874a-8551508b3edf None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:59.937542 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b1f10-59bb-4475-874a-8551508b3edf None admin] RBAC: Authorizing `identity:get_project(project_id=930b378be44b4bb7a2009c72a424f1d6)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:59.939433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b1f10-59bb-4475-874a-8551508b3edf None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:59.941516 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 46/90] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:19:59 2022] GET /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:59.942678 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b1f10-59bb-4475-874a-8551508b3edf None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:19:59.956478 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58a975-f1d8-4ed3-8921-c5a08ee7735e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:19:59.957065 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58a975-f1d8-4ed3-8921-c5a08ee7735e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:19:59.957348 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58a975-f1d8-4ed3-8921-c5a08ee7735e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:19:59.957673 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58a975-f1d8-4ed3-8921-c5a08ee7735e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:19:59.958509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58a975-f1d8-4ed3-8921-c5a08ee7735e None admin] RBAC: Adding query filter params (role.id=5ca112885467422684e134cb0f94865a, scope.project.id=930b378be44b4bb7a2009c72a424f1d6, user.id=8b5c1280f1184855bd78412238939419) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:19:59.958870 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58a975-f1d8-4ed3-8921-c5a08ee7735e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:19:59.960410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58a975-f1d8-4ed3-8921-c5a08ee7735e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:19:59.966675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 45/91] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:19:59 2022] GET /identity/v3/role_assignments?user.id=8b5c1280f1184855bd78412238939419&scope.project.id=930b378be44b4bb7a2009c72a424f1d6&role.id=5ca112885467422684e134cb0f94865a => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:19:59.968216 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58a975-f1d8-4ed3-8921-c5a08ee7735e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:00.043063 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:00.065909 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b1f10-59bb-4475-874a-8551508b3edf None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:00.085411 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58a975-f1d8-4ed3-8921-c5a08ee7735e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:00.107083 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03563fa-72fc-41bf-ba6b-86b44feee5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:00.108974 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:00.110081 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03563fa-72fc-41bf-ba6b-86b44feee5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:00.111018 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03563fa-72fc-41bf-ba6b-86b44feee5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:00.111625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03563fa-72fc-41bf-ba6b-86b44feee5f4 None admin] PATH_INFO: `/v3/users/2f2bd63cb926471a9ba1d25571f4dd51` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:00.114555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03563fa-72fc-41bf-ba6b-86b44feee5f4 None admin] RBAC: Authorizing `identity:get_user(user_id=2f2bd63cb926471a9ba1d25571f4dd51)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:00.116979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03563fa-72fc-41bf-ba6b-86b44feee5f4 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:00.120602 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 47/92] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:19:59 2022] GET /identity/v3/users/2f2bd63cb926471a9ba1d25571f4dd51 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:00.122246 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03563fa-72fc-41bf-ba6b-86b44feee5f4 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:00.130821 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decc1a2-47d8-43ba-a269-61daf5e9c083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:00.131622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decc1a2-47d8-43ba-a269-61daf5e9c083 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:00.131995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decc1a2-47d8-43ba-a269-61daf5e9c083 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:00.132347 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decc1a2-47d8-43ba-a269-61daf5e9c083 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:00.133289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decc1a2-47d8-43ba-a269-61daf5e9c083 None admin] RBAC: Adding query filter params (role.id=b778e530d54948c7bc31f5b9231ea288, scope.project.id=930b378be44b4bb7a2009c72a424f1d6, user.id=2f2bd63cb926471a9ba1d25571f4dd51) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:00.133675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decc1a2-47d8-43ba-a269-61daf5e9c083 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:00.135285 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decc1a2-47d8-43ba-a269-61daf5e9c083 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:00.144021 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 46/93] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:19:59 2022] GET /identity/v3/role_assignments?user.id=2f2bd63cb926471a9ba1d25571f4dd51&scope.project.id=930b378be44b4bb7a2009c72a424f1d6&role.id=b778e530d54948c7bc31f5b9231ea288 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:00.145069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decc1a2-47d8-43ba-a269-61daf5e9c083 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:00.281277 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03563fa-72fc-41bf-ba6b-86b44feee5f4 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:00.313856 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decc1a2-47d8-43ba-a269-61daf5e9c083 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:00.337530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f13a05-3126-47bd-9564-fca6cf205e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:00.338776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f13a05-3126-47bd-9564-fca6cf205e2c None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:00.339284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f13a05-3126-47bd-9564-fca6cf205e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:00.340119 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f13a05-3126-47bd-9564-fca6cf205e2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:00.341596 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f13a05-3126-47bd-9564-fca6cf205e2c None admin] RBAC: Adding query filter params (role.id=6b2eb3e4cefd4a2db4f18e04a158600a, scope.project.id=930b378be44b4bb7a2009c72a424f1d6, user.id=2f2bd63cb926471a9ba1d25571f4dd51) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:00.342120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f13a05-3126-47bd-9564-fca6cf205e2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:00.343826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f13a05-3126-47bd-9564-fca6cf205e2c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:00.353201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 48/94] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:00 2022] GET /identity/v3/role_assignments?user.id=2f2bd63cb926471a9ba1d25571f4dd51&scope.project.id=930b378be44b4bb7a2009c72a424f1d6&role.id=6b2eb3e4cefd4a2db4f18e04a158600a => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:00.385621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8c1b8-f5eb-433b-b2a3-1990dc16e36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:00.386359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8c1b8-f5eb-433b-b2a3-1990dc16e36d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:00.386657 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8c1b8-f5eb-433b-b2a3-1990dc16e36d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:00.386966 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8c1b8-f5eb-433b-b2a3-1990dc16e36d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:00.474988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:00.495406 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd8c1b8-f5eb-433b-b2a3-1990dc16e36d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:00.496901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd8c1b8-f5eb-433b-b2a3-1990dc16e36d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:00.665840 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 47/95] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:00.671289 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f13a05-3126-47bd-9564-fca6cf205e2c None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:00.691537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:00.863092 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f13a05-3126-47bd-9564-fca6cf205e2c None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:00.918777 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241d7a6-9063-40db-a0f0-c37916426922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:00.919621 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241d7a6-9063-40db-a0f0-c37916426922 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:00.919903 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241d7a6-9063-40db-a0f0-c37916426922 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:00.920178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241d7a6-9063-40db-a0f0-c37916426922 None admin] PATH_INFO: `/v3/projects/05a1fc92d37549878f650ecbc4773f33` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:00.928044 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241d7a6-9063-40db-a0f0-c37916426922 None admin] RBAC: Authorizing `identity:get_project(project_id=05a1fc92d37549878f650ecbc4773f33)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:00.929579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241d7a6-9063-40db-a0f0-c37916426922 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:00.931519 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 49/96] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:00 2022] GET /identity/v3/projects/05a1fc92d37549878f650ecbc4773f33 => generated 312 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:00.938297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8c1b8-f5eb-433b-b2a3-1990dc16e36d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:01.082788 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:01.103930 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8c1b8-f5eb-433b-b2a3-1990dc16e36d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:01.161714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb253e-f3a6-4a29-b882-db331e131c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:01.162388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb253e-f3a6-4a29-b882-db331e131c8b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:01.162676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb253e-f3a6-4a29-b882-db331e131c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:01.162965 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb253e-f3a6-4a29-b882-db331e131c8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:01.164082 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb253e-f3a6-4a29-b882-db331e131c8b None admin] RBAC: Adding query filter params (role.id=b778e530d54948c7bc31f5b9231ea288, scope.project.id=05a1fc92d37549878f650ecbc4773f33, user.id=2f2bd63cb926471a9ba1d25571f4dd51) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:01.164396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb253e-f3a6-4a29-b882-db331e131c8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:01.165904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb253e-f3a6-4a29-b882-db331e131c8b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:01.178181 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 48/97] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:00 2022] GET /identity/v3/role_assignments?user.id=2f2bd63cb926471a9ba1d25571f4dd51&scope.project.id=05a1fc92d37549878f650ecbc4773f33&role.id=b778e530d54948c7bc31f5b9231ea288 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:01.532852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87508e-71bd-4c32-804f-74f012b5ed46 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:01.533225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87508e-71bd-4c32-804f-74f012b5ed46 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:01.533417 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87508e-71bd-4c32-804f-74f012b5ed46 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:01.534972 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 50/98] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:01.549877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c1ec9-4891-4ddf-b869-035d4ff27bfb None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:01.550170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c1ec9-4891-4ddf-b869-035d4ff27bfb None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:01.550442 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c1ec9-4891-4ddf-b869-035d4ff27bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:01.580069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-721c1ec9-4891-4ddf-b869-035d4ff27bfb None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:01.581334 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-721c1ec9-4891-4ddf-b869-035d4ff27bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:01.688355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 49/99] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:01.735830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87508e-71bd-4c32-804f-74f012b5ed46 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:01.749072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2c78c-0061-4c58-af10-779fa194496b None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:01.749072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2c78c-0061-4c58-af10-779fa194496b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:01.749072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2c78c-0061-4c58-af10-779fa194496b None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:01.749072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 50/100] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:20:01.754877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cf33b-5ca2-4792-ac52-e6f1894c407f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:01.755491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cf33b-5ca2-4792-ac52-e6f1894c407f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:01.756059 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cf33b-5ca2-4792-ac52-e6f1894c407f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:01.774317 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-681cf33b-5ca2-4792-ac52-e6f1894c407f None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:01.775892 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-681cf33b-5ca2-4792-ac52-e6f1894c407f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:01.851262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 51/101] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:20:01.875416 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cf33b-5ca2-4792-ac52-e6f1894c407f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:01.886122 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:01.895832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:01.928789 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87508e-71bd-4c32-804f-74f012b5ed46 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:01.985282 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cf33b-5ca2-4792-ac52-e6f1894c407f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:01.995510 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01288bff-a066-4192-a6ee-22eb68bd8ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:01.996334 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01288bff-a066-4192-a6ee-22eb68bd8ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:01.996623 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01288bff-a066-4192-a6ee-22eb68bd8ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:01.996903 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01288bff-a066-4192-a6ee-22eb68bd8ab0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:02.017569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01288bff-a066-4192-a6ee-22eb68bd8ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:02.022846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd81b3-ed7f-4fe4-939e-d98b1f157479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:02.023857 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd81b3-ed7f-4fe4-939e-d98b1f157479 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:02.024272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd81b3-ed7f-4fe4-939e-d98b1f157479 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:02.024714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd81b3-ed7f-4fe4-939e-d98b1f157479 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:02.026037 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd81b3-ed7f-4fe4-939e-d98b1f157479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:02.026149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-01288bff-a066-4192-a6ee-22eb68bd8ab0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 20:20:02.026149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:02.026149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:02.026149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:02.026149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:02.026149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:02.026149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:02.026149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:02.026149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:02.026149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:02.026149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:02.026149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:02.026149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:02.026149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:02.026149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:02.026149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:02.027202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:02.027202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:02.027202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:02.027202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:02.027202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:02.027202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:02.027202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:02.027202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:02.027202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:02.027202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:02.027202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:02.027202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:02.027202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:02.027202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:02.027202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:02.027202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:02.027956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:02.027956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:02.027956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:02.027956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:02.027956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:02.027956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:02.027956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:02.027956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:02.027956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:02.027956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:02.027956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 20:20:02.027956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:02.028540 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd81b3-ed7f-4fe4-939e-d98b1f157479 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:02.034291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 52/102] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:02.035522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 51/103] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:02.041618 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd81b3-ed7f-4fe4-939e-d98b1f157479 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:02.041859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01288bff-a066-4192-a6ee-22eb68bd8ab0 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:02.255383 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:02.266472 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01288bff-a066-4192-a6ee-22eb68bd8ab0 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:02.304248 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd81b3-ed7f-4fe4-939e-d98b1f157479 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:02.363689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b96e64-fd18-42ca-852f-8dc1211aad99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:02.364386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b96e64-fd18-42ca-852f-8dc1211aad99 None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:02.364674 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b96e64-fd18-42ca-852f-8dc1211aad99 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:02.364952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b96e64-fd18-42ca-852f-8dc1211aad99 None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:02.365334 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ead11-f088-460d-875c-2239641c3225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:02.365678 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b96e64-fd18-42ca-852f-8dc1211aad99 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:02.366437 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ead11-f088-460d-875c-2239641c3225 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:02.366959 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ead11-f088-460d-875c-2239641c3225 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:02.367172 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b96e64-fd18-42ca-852f-8dc1211aad99 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:02.367633 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ead11-f088-460d-875c-2239641c3225 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:02.368587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ead11-f088-460d-875c-2239641c3225 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:02.369087 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ead11-f088-460d-875c-2239641c3225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:02.370667 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ead11-f088-460d-875c-2239641c3225 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:02.384902 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 53/104] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:02.391273 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ead11-f088-460d-875c-2239641c3225 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:02.395643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:02.423397 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 52/105] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 14 20:20:02 2022] POST /identity/v3/projects => generated 302 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:20:02.556049 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ead11-f088-460d-875c-2239641c3225 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:02.594716 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0d65a-b875-4d27-a91f-9bcbec81110d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:02.596031 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0d65a-b875-4d27-a91f-9bcbec81110d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:02.596516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0d65a-b875-4d27-a91f-9bcbec81110d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:02.597038 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0d65a-b875-4d27-a91f-9bcbec81110d None admin] PATH_INFO: `/v3/users/700a9c53c7624ddc839cbd2a4c2c169f` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:02.606636 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:02.614997 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0d65a-b875-4d27-a91f-9bcbec81110d None admin] RBAC: Authorizing `identity:get_user(user_id=700a9c53c7624ddc839cbd2a4c2c169f)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:02.616946 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0d65a-b875-4d27-a91f-9bcbec81110d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:02.619176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 54/106] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:02 2022] GET /identity/v3/users/700a9c53c7624ddc839cbd2a4c2c169f => generated 289 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:02.626066 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b96e64-fd18-42ca-852f-8dc1211aad99 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:02.794728 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b96e64-fd18-42ca-852f-8dc1211aad99 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:02.835979 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dc55b-bbbc-4228-ad62-7f3602777c49 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:02.836465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dc55b-bbbc-4228-ad62-7f3602777c49 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:02.836465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dc55b-bbbc-4228-ad62-7f3602777c49 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:02.837451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29e31c-18e7-4697-8812-a72d833195ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:02.837836 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 55/107] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:20:02.838123 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29e31c-18e7-4697-8812-a72d833195ac None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:02.838416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29e31c-18e7-4697-8812-a72d833195ac None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:02.838702 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29e31c-18e7-4697-8812-a72d833195ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:02.844271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b93ec-1440-4787-9f01-30c17e19fe00 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:02.844600 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b93ec-1440-4787-9f01-30c17e19fe00 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:02.844869 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b93ec-1440-4787-9f01-30c17e19fe00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:02.884057 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-989b93ec-1440-4787-9f01-30c17e19fe00 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:02.885324 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-989b93ec-1440-4787-9f01-30c17e19fe00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:02.962294 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:02.981573 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29e31c-18e7-4697-8812-a72d833195ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:02.985570 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29e31c-18e7-4697-8812-a72d833195ac None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:03.058309 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 56/108] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:20:03.101286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b93ec-1440-4787-9f01-30c17e19fe00 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:03.162963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 53/109] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:03.167257 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29e31c-18e7-4697-8812-a72d833195ac None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:03.271844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29e31c-18e7-4697-8812-a72d833195ac None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:03.281892 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:03.312538 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedeb1c-dc0b-47d4-ab00-679e31644a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:03.313936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedeb1c-dc0b-47d4-ab00-679e31644a4f None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:03.314532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedeb1c-dc0b-47d4-ab00-679e31644a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:03.315019 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedeb1c-dc0b-47d4-ab00-679e31644a4f None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:03.316649 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bedeb1c-dc0b-47d4-ab00-679e31644a4f None admin] RBAC: Authorizing `identity:get_project(project_id=930b378be44b4bb7a2009c72a424f1d6)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:03.318314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bedeb1c-dc0b-47d4-ab00-679e31644a4f None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:03.320259 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 54/110] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:03 2022] GET /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:03.331620 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedeb1c-dc0b-47d4-ab00-679e31644a4f None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:03.409097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b93ec-1440-4787-9f01-30c17e19fe00 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:03.493074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:03.495272 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedeb1c-dc0b-47d4-ab00-679e31644a4f None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:03.503322 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c8ec27-9ee9-48c1-a80f-f11a249fae98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:03.507886 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8ec27-9ee9-48c1-a80f-f11a249fae98 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:03.508417 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8ec27-9ee9-48c1-a80f-f11a249fae98 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:03.508838 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8ec27-9ee9-48c1-a80f-f11a249fae98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:03.526461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c8ec27-9ee9-48c1-a80f-f11a249fae98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:03.543294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-b7c8ec27-9ee9-48c1-a80f-f11a249fae98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:20:03.543294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:03.543294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:03.543294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:03.543294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:03.543294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:03.543294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:03.543294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:03.543294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:03.543294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:03.543294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:03.543294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:03.543294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:03.543294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:03.543294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:03.543294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:03.544378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:03.544378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:03.544378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:03.544378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:03.544378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:03.544378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:03.544378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:03.544378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:03.544378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:03.544378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:03.544378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:03.544378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:03.544378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:03.544378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:03.544378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:03.544378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:03.545316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:03.545316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:03.545316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:03.545316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:03.545316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:03.545316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:03.545316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:03.545316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:03.545316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:03.545316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:03.545316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:20:03.545316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:03.547350 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 57/111] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:20:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 447 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:03.557894 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c8ec27-9ee9-48c1-a80f-f11a249fae98 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:03.563736 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f891688-756a-4776-a35d-f8090c201d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:03.571019 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f891688-756a-4776-a35d-f8090c201d9d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:03.571019 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f891688-756a-4776-a35d-f8090c201d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:03.571019 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f891688-756a-4776-a35d-f8090c201d9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:03.571019 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f891688-756a-4776-a35d-f8090c201d9d None admin] RBAC: Adding query filter params (role.id=a7d155dd59914d22a32012302fa6d2d2, scope.project.id=930b378be44b4bb7a2009c72a424f1d6, user.id=700a9c53c7624ddc839cbd2a4c2c169f) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:03.571019 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f891688-756a-4776-a35d-f8090c201d9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:03.571019 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f891688-756a-4776-a35d-f8090c201d9d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:03.579099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 55/112] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:03 2022] GET /identity/v3/role_assignments?user.id=700a9c53c7624ddc839cbd2a4c2c169f&scope.project.id=930b378be44b4bb7a2009c72a424f1d6&role.id=a7d155dd59914d22a32012302fa6d2d2 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:03.727022 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c8ec27-9ee9-48c1-a80f-f11a249fae98 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:03.748675 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048764ae-6181-4a9d-8266-db7a28a92c3c None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:03.749208 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048764ae-6181-4a9d-8266-db7a28a92c3c None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:03.749634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048764ae-6181-4a9d-8266-db7a28a92c3c None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:03.751306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 56/113] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:03.760488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d0e53-58ee-4300-b0bf-5a130125d0fb None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:03.761135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d0e53-58ee-4300-b0bf-5a130125d0fb None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:03.761612 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d0e53-58ee-4300-b0bf-5a130125d0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:03.771914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f39b63-64e9-4569-a931-1f45a1873fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:03.772644 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f39b63-64e9-4569-a931-1f45a1873fed None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:03.772935 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f39b63-64e9-4569-a931-1f45a1873fed None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:03.773219 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f39b63-64e9-4569-a931-1f45a1873fed None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:03.774022 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f39b63-64e9-4569-a931-1f45a1873fed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:03.778976 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f39b63-64e9-4569-a931-1f45a1873fed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:03.778976 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f39b63-64e9-4569-a931-1f45a1873fed None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:03.787045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 58/114] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:03.794965 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f39b63-64e9-4569-a931-1f45a1873fed None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:03.804136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-6d2d0e53-58ee-4300-b0bf-5a130125d0fb None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:03.805752 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6d2d0e53-58ee-4300-b0bf-5a130125d0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:03.854780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:03.879338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:03.918480 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f39b63-64e9-4569-a931-1f45a1873fed None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:03.967936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcab5a-7b6e-4b38-8f97-88946cf7ea68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:03.969647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcab5a-7b6e-4b38-8f97-88946cf7ea68 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:03.970107 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcab5a-7b6e-4b38-8f97-88946cf7ea68 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:03.970529 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcab5a-7b6e-4b38-8f97-88946cf7ea68 None admin] PATH_INFO: `/v3/users/8b5c1280f1184855bd78412238939419` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:03.972122 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fcab5a-7b6e-4b38-8f97-88946cf7ea68 None admin] RBAC: Authorizing `identity:get_user(user_id=8b5c1280f1184855bd78412238939419)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:03.974420 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fcab5a-7b6e-4b38-8f97-88946cf7ea68 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:03.976939 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 59/115] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:03 2022] GET /identity/v3/users/8b5c1280f1184855bd78412238939419 => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:03.984323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcab5a-7b6e-4b38-8f97-88946cf7ea68 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:04.002266 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 57/116] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:04.052619 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d0e53-58ee-4300-b0bf-5a130125d0fb None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:04.172882 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcab5a-7b6e-4b38-8f97-88946cf7ea68 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:04.177595 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d0e53-58ee-4300-b0bf-5a130125d0fb None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:04.191422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:04.210408 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c2151-54d6-4622-a440-91c67152e166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:04.211764 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c2151-54d6-4622-a440-91c67152e166 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:04.212246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c2151-54d6-4622-a440-91c67152e166 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:04.212695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c2151-54d6-4622-a440-91c67152e166 None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:04.214107 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c2151-54d6-4622-a440-91c67152e166 None admin] RBAC: Authorizing `identity:get_project(project_id=930b378be44b4bb7a2009c72a424f1d6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:04.215790 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c2151-54d6-4622-a440-91c67152e166 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:04.217739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 60/117] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:03 2022] GET /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:04.219537 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54731aba-edcc-4a63-a852-960b9db27ff3 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:04.220118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54731aba-edcc-4a63-a852-960b9db27ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:04.220573 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54731aba-edcc-4a63-a852-960b9db27ff3 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:04.221508 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8e928-61c6-47dd-bcc6-5dad3f429449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:04.222469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8e928-61c6-47dd-bcc6-5dad3f429449 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:04.222827 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8e928-61c6-47dd-bcc6-5dad3f429449 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:04.222926 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 61/118] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:04.223204 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8e928-61c6-47dd-bcc6-5dad3f429449 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:04.224568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9d3c3-d235-48a2-ba46-e2fb31a646f4 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:04.225031 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9d3c3-d235-48a2-ba46-e2fb31a646f4 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:04.225475 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9d3c3-d235-48a2-ba46-e2fb31a646f4 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:04.227068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 62/119] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:04.227984 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9d3c3-d235-48a2-ba46-e2fb31a646f4 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:04.237653 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c8e928-61c6-47dd-bcc6-5dad3f429449 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:04.246021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-90c8e928-61c6-47dd-bcc6-5dad3f429449 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:04.246021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:04.246021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:04.246021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:04.246021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:04.246021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:04.246021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:04.246021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:04.246021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:04.246021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:04.246021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:04.246021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:04.246021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:04.246021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:04.246021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:04.246021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:04.246902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:04.246902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:04.246902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:04.246902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:04.246902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:04.246902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:04.246902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:04.246902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:04.246902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:04.246902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:04.246902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:04.246902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:04.246902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:04.246902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:04.246902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:04.246902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:04.247693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:04.247693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:04.247693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:04.247693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:04.247693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:04.247693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:04.247693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:04.247693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:04.247693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:04.247693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:04.247693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:04.247693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:04.247693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 58/120] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:04.253936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b271fc7-de1c-4d3a-a9e2-4ccccbda5cbb None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:04.253936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b271fc7-de1c-4d3a-a9e2-4ccccbda5cbb None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:04.253936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b271fc7-de1c-4d3a-a9e2-4ccccbda5cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:04.267666 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-8b271fc7-de1c-4d3a-a9e2-4ccccbda5cbb None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:04.269238 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-8b271fc7-de1c-4d3a-a9e2-4ccccbda5cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:04.282039 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:04.347267 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9d3c3-d235-48a2-ba46-e2fb31a646f4 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:04.348150 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 59/121] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:04.349670 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5ed14-1a4a-4606-9f59-5533232781c4 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:04.350034 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5ed14-1a4a-4606-9f59-5533232781c4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:04.350391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5ed14-1a4a-4606-9f59-5533232781c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:04.369025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-22b5ed14-1a4a-4606-9f59-5533232781c4 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:04.370331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-22b5ed14-1a4a-4606-9f59-5533232781c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:04.387433 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17a2ae-7d7a-4a2a-bb19-eb43b02b34f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:04.388699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17a2ae-7d7a-4a2a-bb19-eb43b02b34f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:04.389166 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17a2ae-7d7a-4a2a-bb19-eb43b02b34f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:04.389656 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17a2ae-7d7a-4a2a-bb19-eb43b02b34f4 None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6/users/8b5c1280f1184855bd78412238939419/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:04.392070 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17a2ae-7d7a-4a2a-bb19-eb43b02b34f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=930b378be44b4bb7a2009c72a424f1d6, user_id=8b5c1280f1184855bd78412238939419, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:04.394028 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17a2ae-7d7a-4a2a-bb19-eb43b02b34f4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:04.411951 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 63/122] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:20:04 2022] PUT /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6/users/8b5c1280f1184855bd78412238939419/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:20:04.413273 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17a2ae-7d7a-4a2a-bb19-eb43b02b34f4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:04.466076 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:04.466986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 60/123] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:04.468207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5ed14-1a4a-4606-9f59-5533232781c4 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:04.528952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17a2ae-7d7a-4a2a-bb19-eb43b02b34f4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:04.559251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:04.568895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9788fb-a16e-4ce0-bd9d-f43512e3354d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:04.569935 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9788fb-a16e-4ce0-bd9d-f43512e3354d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:04.570420 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9788fb-a16e-4ce0-bd9d-f43512e3354d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:04.570893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9788fb-a16e-4ce0-bd9d-f43512e3354d None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:04.571918 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9788fb-a16e-4ce0-bd9d-f43512e3354d None admin] RBAC: Adding query filter params (name=member) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:04.572422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9788fb-a16e-4ce0-bd9d-f43512e3354d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:04.573996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9788fb-a16e-4ce0-bd9d-f43512e3354d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:04.583503 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 64/124] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:04.587453 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5ed14-1a4a-4606-9f59-5533232781c4 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:04.588446 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9788fb-a16e-4ce0-bd9d-f43512e3354d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:04.629129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf976d-1766-4c3c-bd72-5c4169a499ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:04.629129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf976d-1766-4c3c-bd72-5c4169a499ca None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:04.629129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf976d-1766-4c3c-bd72-5c4169a499ca None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:04.629129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf976d-1766-4c3c-bd72-5c4169a499ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:04.641935 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf976d-1766-4c3c-bd72-5c4169a499ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:04.650882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-17bf976d-1766-4c3c-bd72-5c4169a499ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:04.650882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:04.650882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:04.650882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:04.650882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:04.650882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:04.650882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:04.650882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:04.650882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:04.650882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:04.650882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:04.650882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:04.650882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:04.650882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:04.650882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:04.650882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:04.651864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:04.651864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:04.651864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:04.651864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:04.651864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:04.651864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:04.651864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:04.651864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:04.651864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:04.651864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:04.651864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:04.651864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:04.651864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:04.651864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:04.651864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:04.651864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:04.653079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:04.653079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:04.653079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:04.653079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:04.653079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:04.653079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:04.653079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:04.653079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:04.653079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:04.653079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:04.653079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:04.653079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:04.653079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 61/125] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:20:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:04.653944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf976d-1766-4c3c-bd72-5c4169a499ca None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:04.720986 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9788fb-a16e-4ce0-bd9d-f43512e3354d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:04.728874 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:04.763232 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a411b-d602-44a6-ada0-92b581ad02a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:04.764443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a411b-d602-44a6-ada0-92b581ad02a4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:04.764962 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a411b-d602-44a6-ada0-92b581ad02a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:04.765434 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a411b-d602-44a6-ada0-92b581ad02a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:04.777671 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a411b-d602-44a6-ada0-92b581ad02a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:04.787380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-4d6a411b-d602-44a6-ada0-92b581ad02a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:04.787380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:04.787380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:04.787380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:04.787380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:04.787380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:04.787380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:04.787380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:04.787380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:04.787380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:04.787380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:04.787380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:04.787380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:04.787380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:04.787380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:04.787380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:04.788234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:04.788234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:04.788234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:04.788234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:04.788234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:04.788234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:04.788234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:04.788234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:04.788234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:04.788234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:04.788234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:04.788234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:04.788234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:04.788234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:04.788234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:04.788234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:04.788996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:04.788996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:04.788996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:04.788996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:04.788996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:04.788996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:04.788996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:04.788996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:04.788996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:04.788996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:04.788996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:04.788996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:04.791708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 65/126] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:04.792783 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a411b-d602-44a6-ada0-92b581ad02a4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:04.800605 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf976d-1766-4c3c-bd72-5c4169a499ca None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:04.838579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ee0e7-d68e-4f9d-bb16-37053f90de8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:04.839571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ee0e7-d68e-4f9d-bb16-37053f90de8a None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:04.839865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ee0e7-d68e-4f9d-bb16-37053f90de8a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:04.840150 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ee0e7-d68e-4f9d-bb16-37053f90de8a None admin] PATH_INFO: `/v3/users/2f2bd63cb926471a9ba1d25571f4dd51` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:04.841429 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ee0e7-d68e-4f9d-bb16-37053f90de8a None admin] RBAC: Authorizing `identity:get_user(user_id=2f2bd63cb926471a9ba1d25571f4dd51)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:04.843784 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ee0e7-d68e-4f9d-bb16-37053f90de8a None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:04.846524 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 62/127] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:04 2022] GET /identity/v3/users/2f2bd63cb926471a9ba1d25571f4dd51 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:04.847483 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ee0e7-d68e-4f9d-bb16-37053f90de8a None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:04.852011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:04.908497 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a411b-d602-44a6-ada0-92b581ad02a4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:04.948608 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe771b45-fcb4-4f87-bbe1-b7779291842b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:04.949594 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe771b45-fcb4-4f87-bbe1-b7779291842b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:04.950026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe771b45-fcb4-4f87-bbe1-b7779291842b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:04.951165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe771b45-fcb4-4f87-bbe1-b7779291842b None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:04.952913 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe771b45-fcb4-4f87-bbe1-b7779291842b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:04.953404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe771b45-fcb4-4f87-bbe1-b7779291842b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:04.955808 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe771b45-fcb4-4f87-bbe1-b7779291842b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:04.965683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ee0e7-d68e-4f9d-bb16-37053f90de8a None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:04.968301 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 66/128] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:20:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:04.970241 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe771b45-fcb4-4f87-bbe1-b7779291842b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:05.005579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:05.012175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76fa60-3ecf-4a15-8380-7b9f65b10a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:05.013384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76fa60-3ecf-4a15-8380-7b9f65b10a97 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:05.013817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76fa60-3ecf-4a15-8380-7b9f65b10a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:05.014764 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76fa60-3ecf-4a15-8380-7b9f65b10a97 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:05.015762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76fa60-3ecf-4a15-8380-7b9f65b10a97 None admin] RBAC: Adding query filter params (name=member) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:05.016227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76fa60-3ecf-4a15-8380-7b9f65b10a97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:05.017796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76fa60-3ecf-4a15-8380-7b9f65b10a97 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:05.028757 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 63/129] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:05.029952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76fa60-3ecf-4a15-8380-7b9f65b10a97 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:05.087951 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe771b45-fcb4-4f87-bbe1-b7779291842b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:05.106446 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:05.133116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0f0f5-0bb6-4386-bfc7-e15e83bace6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:05.134411 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0f0f5-0bb6-4386-bfc7-e15e83bace6f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:05.134411 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0f0f5-0bb6-4386-bfc7-e15e83bace6f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:05.134576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0f0f5-0bb6-4386-bfc7-e15e83bace6f None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:05.140303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0f0f5-0bb6-4386-bfc7-e15e83bace6f None admin] RBAC: Authorizing `identity:get_project(project_id=930b378be44b4bb7a2009c72a424f1d6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:05.141820 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0f0f5-0bb6-4386-bfc7-e15e83bace6f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:05.143908 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 67/130] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:04 2022] GET /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:05.148336 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0f0f5-0bb6-4386-bfc7-e15e83bace6f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:05.149696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76fa60-3ecf-4a15-8380-7b9f65b10a97 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:05.186478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901328c-3cdd-430d-ae48-5933e3bf7889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:05.187828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901328c-3cdd-430d-ae48-5933e3bf7889 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:05.188407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901328c-3cdd-430d-ae48-5933e3bf7889 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:05.189160 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901328c-3cdd-430d-ae48-5933e3bf7889 None admin] PATH_INFO: `/v3/users/2f2bd63cb926471a9ba1d25571f4dd51` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:05.190683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901328c-3cdd-430d-ae48-5933e3bf7889 None admin] RBAC: Authorizing `identity:get_user(user_id=2f2bd63cb926471a9ba1d25571f4dd51)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:05.192369 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901328c-3cdd-430d-ae48-5933e3bf7889 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:05.194644 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 64/131] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:05 2022] GET /identity/v3/users/2f2bd63cb926471a9ba1d25571f4dd51 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:05.195627 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901328c-3cdd-430d-ae48-5933e3bf7889 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:05.304122 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901328c-3cdd-430d-ae48-5933e3bf7889 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:05.323345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:05.328068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0f0f5-0bb6-4386-bfc7-e15e83bace6f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:05.365610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e49e7-8e54-4518-867b-3e460233fbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:05.366699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e49e7-8e54-4518-867b-3e460233fbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:05.367026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e49e7-8e54-4518-867b-3e460233fbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:05.367321 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e49e7-8e54-4518-867b-3e460233fbe1 None admin] PATH_INFO: `/v3/users/2f2bd63cb926471a9ba1d25571f4dd51` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:05.369044 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e49e7-8e54-4518-867b-3e460233fbe1 None admin] RBAC: Authorizing `identity:get_user(user_id=2f2bd63cb926471a9ba1d25571f4dd51)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:05.370531 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e49e7-8e54-4518-867b-3e460233fbe1 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:05.372725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 68/132] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:05 2022] GET /identity/v3/users/2f2bd63cb926471a9ba1d25571f4dd51 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:05.374334 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e49e7-8e54-4518-867b-3e460233fbe1 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:05.379075 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25de6b4-2a57-430b-8e61-2c705094fb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:05.380025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25de6b4-2a57-430b-8e61-2c705094fb7f None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:05.380313 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25de6b4-2a57-430b-8e61-2c705094fb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:05.380593 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25de6b4-2a57-430b-8e61-2c705094fb7f None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6/users/2f2bd63cb926471a9ba1d25571f4dd51/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:05.388433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25de6b4-2a57-430b-8e61-2c705094fb7f None admin] RBAC: Authorizing `identity:create_grant(project_id=930b378be44b4bb7a2009c72a424f1d6, user_id=2f2bd63cb926471a9ba1d25571f4dd51, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:05.392433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25de6b4-2a57-430b-8e61-2c705094fb7f None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:05.399297 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:05.417298 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 65/133] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:20:05 2022] PUT /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6/users/2f2bd63cb926471a9ba1d25571f4dd51/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:20:05.418380 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25de6b4-2a57-430b-8e61-2c705094fb7f None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:05.553278 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25de6b4-2a57-430b-8e61-2c705094fb7f None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:05.593042 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f4b89-2656-4c12-9363-a85a51681b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:05.594032 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f4b89-2656-4c12-9363-a85a51681b49 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:05.594318 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f4b89-2656-4c12-9363-a85a51681b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:05.594671 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f4b89-2656-4c12-9363-a85a51681b49 None admin] PATH_INFO: `/v3/projects/05a1fc92d37549878f650ecbc4773f33` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:05.598971 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f4b89-2656-4c12-9363-a85a51681b49 None admin] RBAC: Authorizing `identity:get_project(project_id=05a1fc92d37549878f650ecbc4773f33)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:05.599276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f4b89-2656-4c12-9363-a85a51681b49 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:05.602232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 66/134] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:05 2022] GET /identity/v3/projects/05a1fc92d37549878f650ecbc4773f33 => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:05.608624 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f4b89-2656-4c12-9363-a85a51681b49 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:05.613348 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e49e7-8e54-4518-867b-3e460233fbe1 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:05.695673 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:05.711750 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed4df3-5ab8-4be5-b75e-9ccbabd0387d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:05.713030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed4df3-5ab8-4be5-b75e-9ccbabd0387d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:05.713506 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed4df3-5ab8-4be5-b75e-9ccbabd0387d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:05.713970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed4df3-5ab8-4be5-b75e-9ccbabd0387d None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:05.715496 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed4df3-5ab8-4be5-b75e-9ccbabd0387d None admin] RBAC: Authorizing `identity:get_project(project_id=930b378be44b4bb7a2009c72a424f1d6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:05.717181 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed4df3-5ab8-4be5-b75e-9ccbabd0387d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:05.719172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 69/135] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:05 2022] GET /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6 => generated 298 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:05.724748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed4df3-5ab8-4be5-b75e-9ccbabd0387d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:05.741142 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f4b89-2656-4c12-9363-a85a51681b49 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:05.787819 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684a4d2-aece-493b-8346-ab2ebe83250d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:05.788747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684a4d2-aece-493b-8346-ab2ebe83250d None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:05.790249 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684a4d2-aece-493b-8346-ab2ebe83250d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:05.790249 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684a4d2-aece-493b-8346-ab2ebe83250d None admin] PATH_INFO: `/v3/projects/05a1fc92d37549878f650ecbc4773f33/users/2f2bd63cb926471a9ba1d25571f4dd51/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:05.791762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684a4d2-aece-493b-8346-ab2ebe83250d None admin] RBAC: Authorizing `identity:create_grant(project_id=05a1fc92d37549878f650ecbc4773f33, user_id=2f2bd63cb926471a9ba1d25571f4dd51, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:05.793438 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684a4d2-aece-493b-8346-ab2ebe83250d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:05.805034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:05.822428 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 67/136] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:20:05 2022] PUT /identity/v3/projects/05a1fc92d37549878f650ecbc4773f33/users/2f2bd63cb926471a9ba1d25571f4dd51/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:20:05.899334 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed4df3-5ab8-4be5-b75e-9ccbabd0387d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:05.940757 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0212d6e-f6f5-46bf-b4f8-ce3780948e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:05.941730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0212d6e-f6f5-46bf-b4f8-ce3780948e12 None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:05.942023 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0212d6e-f6f5-46bf-b4f8-ce3780948e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:05.942291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0212d6e-f6f5-46bf-b4f8-ce3780948e12 None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6/users/2f2bd63cb926471a9ba1d25571f4dd51/roles/6b2eb3e4cefd4a2db4f18e04a158600a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:05.944626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0212d6e-f6f5-46bf-b4f8-ce3780948e12 None admin] RBAC: Authorizing `identity:create_grant(project_id=930b378be44b4bb7a2009c72a424f1d6, user_id=2f2bd63cb926471a9ba1d25571f4dd51, role_id=6b2eb3e4cefd4a2db4f18e04a158600a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:05.946082 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0212d6e-f6f5-46bf-b4f8-ce3780948e12 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:05.952810 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cc36f-41e8-41fc-b4e1-6d73bfe763c9 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:05.953735 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cc36f-41e8-41fc-b4e1-6d73bfe763c9 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:05.954253 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cc36f-41e8-41fc-b4e1-6d73bfe763c9 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:05.956276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 68/137] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:20:05.965243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 70/138] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:20:05 2022] PUT /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6/users/2f2bd63cb926471a9ba1d25571f4dd51/roles/6b2eb3e4cefd4a2db4f18e04a158600a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:20:05.978834 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062ef83-a816-44c7-81ba-d81c225380a9 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:05.979539 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062ef83-a816-44c7-81ba-d81c225380a9 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:05.980072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062ef83-a816-44c7-81ba-d81c225380a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:06.006388 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-c062ef83-a816-44c7-81ba-d81c225380a9 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:06.008068 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-c062ef83-a816-44c7-81ba-d81c225380a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:06.112722 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 69/139] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:06.158035 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0212d6e-f6f5-46bf-b4f8-ce3780948e12 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:06.269837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:06.284412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0212d6e-f6f5-46bf-b4f8-ce3780948e12 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:06.326593 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26abfd-f251-43bc-bbca-c1539e80ba27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:06.327494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26abfd-f251-43bc-bbca-c1539e80ba27 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:06.327786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26abfd-f251-43bc-bbca-c1539e80ba27 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:06.328083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26abfd-f251-43bc-bbca-c1539e80ba27 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:06.329373 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26abfd-f251-43bc-bbca-c1539e80ba27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:06.330877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26abfd-f251-43bc-bbca-c1539e80ba27 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:06.332931 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 71/140] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:06.338637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062ef83-a816-44c7-81ba-d81c225380a9 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:06.476586 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:06.538837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062ef83-a816-44c7-81ba-d81c225380a9 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:06.588605 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb36cc-3898-44a9-a008-4feb74ff3d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:06.589275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320bfb9-cd39-45bc-a627-e79ceddbb4c8 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:06.589401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb36cc-3898-44a9-a008-4feb74ff3d1b None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:06.589570 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320bfb9-cd39-45bc-a627-e79ceddbb4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:06.589663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb36cc-3898-44a9-a008-4feb74ff3d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:06.589842 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb36cc-3898-44a9-a008-4feb74ff3d1b None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:06.589939 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320bfb9-cd39-45bc-a627-e79ceddbb4c8 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:06.590569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb36cc-3898-44a9-a008-4feb74ff3d1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:06.591544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 72/141] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:06.592113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb36cc-3898-44a9-a008-4feb74ff3d1b None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:06.597275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3260bb1-fda4-466d-bdfb-c807a01cf3a8 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:06.597570 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3260bb1-fda4-466d-bdfb-c807a01cf3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:06.597859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3260bb1-fda4-466d-bdfb-c807a01cf3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:06.620571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 70/142] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 14 20:20:06 2022] POST /identity/v3/users => generated 283 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:20:06.632293 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e3260bb1-fda4-466d-bdfb-c807a01cf3a8 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:06.633697 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e3260bb1-fda4-466d-bdfb-c807a01cf3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:06.750318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 73/143] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:20:06.802131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb36cc-3898-44a9-a008-4feb74ff3d1b None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:06.945801 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb36cc-3898-44a9-a008-4feb74ff3d1b None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:06.956628 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:06.984942 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690229b-1fdc-4a3a-a73b-b5c3b3422aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:06.986128 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690229b-1fdc-4a3a-a73b-b5c3b3422aba None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:06.986658 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690229b-1fdc-4a3a-a73b-b5c3b3422aba None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:06.987189 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690229b-1fdc-4a3a-a73b-b5c3b3422aba None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:06.999876 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690229b-1fdc-4a3a-a73b-b5c3b3422aba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:07.008432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-1690229b-1fdc-4a3a-a73b-b5c3b3422aba None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 20:20:07.008432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:07.008432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:07.008432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:07.008432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:07.008432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:07.008432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:07.008432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:07.008432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:07.008432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:07.008432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:07.008432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:07.008432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:07.008432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:07.008432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:07.008432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:07.009357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:07.009357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:07.009357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:07.009357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:07.009357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:07.009357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:07.009357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:07.009357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:07.009357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:07.009357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:07.009357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:07.009357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:07.009357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:07.009357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:07.009357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:07.009357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:07.010152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:07.010152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:07.010152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:07.010152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:07.010152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:07.010152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:07.010152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:07.010152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:07.010152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:07.010152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:07.010152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 20:20:07.010152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:07.012716 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 71/144] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:07.019179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3260bb1-fda4-466d-bdfb-c807a01cf3a8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:07.126009 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:07.138024 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3260bb1-fda4-466d-bdfb-c807a01cf3a8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:07.183308 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22d25d-62a0-48fe-8eff-dc2a96316a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:07.184390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22d25d-62a0-48fe-8eff-dc2a96316a23 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:07.185025 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22d25d-62a0-48fe-8eff-dc2a96316a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:07.185517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22d25d-62a0-48fe-8eff-dc2a96316a23 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:07.186595 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22d25d-62a0-48fe-8eff-dc2a96316a23 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:07.187152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22d25d-62a0-48fe-8eff-dc2a96316a23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:07.188785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22d25d-62a0-48fe-8eff-dc2a96316a23 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:07.200119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 74/145] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:07.206407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690229b-1fdc-4a3a-a73b-b5c3b3422aba None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:07.341200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690229b-1fdc-4a3a-a73b-b5c3b3422aba None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:07.383954 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204537fa-2e7a-41d0-bd72-4ddced257bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:07.385226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204537fa-2e7a-41d0-bd72-4ddced257bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:07.385742 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204537fa-2e7a-41d0-bd72-4ddced257bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:07.386247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204537fa-2e7a-41d0-bd72-4ddced257bf1 None admin] PATH_INFO: `/v3/users/700a9c53c7624ddc839cbd2a4c2c169f` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:07.391702 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204537fa-2e7a-41d0-bd72-4ddced257bf1 None admin] RBAC: Authorizing `identity:get_user(user_id=700a9c53c7624ddc839cbd2a4c2c169f)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:07.393856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204537fa-2e7a-41d0-bd72-4ddced257bf1 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:07.395773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 72/146] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:07 2022] GET /identity/v3/users/700a9c53c7624ddc839cbd2a4c2c169f => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:07.401403 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22d25d-62a0-48fe-8eff-dc2a96316a23 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:07.520118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22d25d-62a0-48fe-8eff-dc2a96316a23 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:07.563724 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a9ce2-0dcf-48e3-b5cd-8bdb716caba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:07.565034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a9ce2-0dcf-48e3-b5cd-8bdb716caba3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:07.565550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a9ce2-0dcf-48e3-b5cd-8bdb716caba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:07.566116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a9ce2-0dcf-48e3-b5cd-8bdb716caba3 None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:07.567579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a9ce2-0dcf-48e3-b5cd-8bdb716caba3 None admin] RBAC: Authorizing `identity:get_project(project_id=930b378be44b4bb7a2009c72a424f1d6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:07.569243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a9ce2-0dcf-48e3-b5cd-8bdb716caba3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:07.571259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 75/147] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:07 2022] GET /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:07.580003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204537fa-2e7a-41d0-bd72-4ddced257bf1 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:07.645276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:07.733647 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204537fa-2e7a-41d0-bd72-4ddced257bf1 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:07.783006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a4f82-1466-49d3-996e-7a8a87bf7f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:07.784386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a4f82-1466-49d3-996e-7a8a87bf7f3a None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:07.784961 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a4f82-1466-49d3-996e-7a8a87bf7f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:07.785459 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a4f82-1466-49d3-996e-7a8a87bf7f3a None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6/users/700a9c53c7624ddc839cbd2a4c2c169f/roles/a7d155dd59914d22a32012302fa6d2d2` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:07.792739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5a4f82-1466-49d3-996e-7a8a87bf7f3a None admin] RBAC: Authorizing `identity:create_grant(project_id=930b378be44b4bb7a2009c72a424f1d6, user_id=700a9c53c7624ddc839cbd2a4c2c169f, role_id=a7d155dd59914d22a32012302fa6d2d2)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:07.794659 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5a4f82-1466-49d3-996e-7a8a87bf7f3a None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:07.814194 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 73/148] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:20:07 2022] PUT /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6/users/700a9c53c7624ddc839cbd2a4c2c169f/roles/a7d155dd59914d22a32012302fa6d2d2 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:20:07.820517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc54e6b-039c-4d74-9cf9-222fac8137cf None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:07.821163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc54e6b-039c-4d74-9cf9-222fac8137cf None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:07.821683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc54e6b-039c-4d74-9cf9-222fac8137cf None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:07.823495 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 76/149] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:07.830379 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4f5b9-d4a6-4301-8b54-83baebeb6738 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:07.831391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4f5b9-d4a6-4301-8b54-83baebeb6738 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:07.832181 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4f5b9-d4a6-4301-8b54-83baebeb6738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:07.864201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-6aa4f5b9-d4a6-4301-8b54-83baebeb6738 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:07.866169 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6aa4f5b9-d4a6-4301-8b54-83baebeb6738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:07.959714 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 74/150] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:08.000782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc54e6b-039c-4d74-9cf9-222fac8137cf None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:08.026508 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:08.128863 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc54e6b-039c-4d74-9cf9-222fac8137cf None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:08.170121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e48d2-b7c2-4fb4-b217-0e9e40665694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:08.171375 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e48d2-b7c2-4fb4-b217-0e9e40665694 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:08.171880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e48d2-b7c2-4fb4-b217-0e9e40665694 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:08.172373 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e48d2-b7c2-4fb4-b217-0e9e40665694 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:08.189106 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e48d2-b7c2-4fb4-b217-0e9e40665694 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:08.201349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-660e48d2-b7c2-4fb4-b217-0e9e40665694 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:20:08.201349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:08.201349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:08.201349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:08.201349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:08.201349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:08.201349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:08.201349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:08.201349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:08.201349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:08.201349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:08.201349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:08.201349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:08.201349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:08.201349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:08.201349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:08.202835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:08.202835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:08.202835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:08.202835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:08.202835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:08.202835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:08.202835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:08.202835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:08.202835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:08.202835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:08.202835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:08.202835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:08.202835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:08.202835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:08.202835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:08.202835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:08.203709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:08.203709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:08.203709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:08.203709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:08.203709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:08.203709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:08.203709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:08.203709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:08.203709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:08.203709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:08.203709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:20:08.203709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:08.205825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 77/151] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:20:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:08.211720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4f5b9-d4a6-4301-8b54-83baebeb6738 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:08.242641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:08.326503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4f5b9-d4a6-4301-8b54-83baebeb6738 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:08.363870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a949f67-edc6-468f-8961-2b42aa3145d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:08.364621 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a949f67-edc6-468f-8961-2b42aa3145d5 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:08.364960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a949f67-edc6-468f-8961-2b42aa3145d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:08.365248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a949f67-edc6-468f-8961-2b42aa3145d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:08.366024 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a949f67-edc6-468f-8961-2b42aa3145d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:08.366335 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a949f67-edc6-468f-8961-2b42aa3145d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:08.367772 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a949f67-edc6-468f-8961-2b42aa3145d5 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:08.377812 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 75/152] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:08.383903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e48d2-b7c2-4fb4-b217-0e9e40665694 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:08.476154 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:08.499240 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e48d2-b7c2-4fb4-b217-0e9e40665694 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:08.540696 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a25c3-c875-498f-92f1-e93f16e1971b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:08.541593 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a25c3-c875-498f-92f1-e93f16e1971b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:08.541877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a25c3-c875-498f-92f1-e93f16e1971b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:08.542153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a25c3-c875-498f-92f1-e93f16e1971b None admin] PATH_INFO: `/v3/users/8b5c1280f1184855bd78412238939419` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:08.543423 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941a25c3-c875-498f-92f1-e93f16e1971b None admin] RBAC: Authorizing `identity:get_user(user_id=8b5c1280f1184855bd78412238939419)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:08.544902 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941a25c3-c875-498f-92f1-e93f16e1971b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:08.546652 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 78/153] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:08 2022] GET /identity/v3/users/8b5c1280f1184855bd78412238939419 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:08.552503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a949f67-edc6-468f-8961-2b42aa3145d5 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:08.655349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:08.669147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a949f67-edc6-468f-8961-2b42aa3145d5 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:08.707150 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08edc93-74ee-419c-894f-da544a97f3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:08.707849 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08edc93-74ee-419c-894f-da544a97f3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:08.708140 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08edc93-74ee-419c-894f-da544a97f3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:08.708509 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08edc93-74ee-419c-894f-da544a97f3e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:08.782635 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08edc93-74ee-419c-894f-da544a97f3e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:08.784234 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08edc93-74ee-419c-894f-da544a97f3e3 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:08.882027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:08.898243 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 76/154] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 20:20:08.903851 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a25c3-c875-498f-92f1-e93f16e1971b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:09.036745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a25c3-c875-498f-92f1-e93f16e1971b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:09.076514 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6b726-d80e-4dcd-9bf2-4a811c51ef3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:09.078661 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6b726-d80e-4dcd-9bf2-4a811c51ef3b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:09.078661 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6b726-d80e-4dcd-9bf2-4a811c51ef3b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:09.078661 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6b726-d80e-4dcd-9bf2-4a811c51ef3b None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:09.080739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6b726-d80e-4dcd-9bf2-4a811c51ef3b None admin] RBAC: Authorizing `identity:get_project(project_id=930b378be44b4bb7a2009c72a424f1d6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:09.080921 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6b726-d80e-4dcd-9bf2-4a811c51ef3b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:09.082562 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 79/155] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:08 2022] GET /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:09.088580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08edc93-74ee-419c-894f-da544a97f3e3 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:09.226441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08edc93-74ee-419c-894f-da544a97f3e3 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:09.244562 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd3549-0a60-48b8-a4a8-06bb7199045d None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:09.244879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd3549-0a60-48b8-a4a8-06bb7199045d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:09.245119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd3549-0a60-48b8-a4a8-06bb7199045d None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:09.246523 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 80/156] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:09.260459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24c76d-db85-4f6c-8ab8-5dc101545dc9 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:09.260816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24c76d-db85-4f6c-8ab8-5dc101545dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:09.261014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24c76d-db85-4f6c-8ab8-5dc101545dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:09.268104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061e503-7146-49d2-9afc-4ad3d1a44170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:09.269147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061e503-7146-49d2-9afc-4ad3d1a44170 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:09.269435 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061e503-7146-49d2-9afc-4ad3d1a44170 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:09.269713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061e503-7146-49d2-9afc-4ad3d1a44170 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:09.270726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061e503-7146-49d2-9afc-4ad3d1a44170 None admin] RBAC: Adding query filter params (role.id=5ca112885467422684e134cb0f94865a, scope.project.id=930b378be44b4bb7a2009c72a424f1d6, user.id=8b5c1280f1184855bd78412238939419) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:09.271090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061e503-7146-49d2-9afc-4ad3d1a44170 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:09.272811 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061e503-7146-49d2-9afc-4ad3d1a44170 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:09.281360 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 77/157] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:09 2022] GET /identity/v3/role_assignments?user.id=8b5c1280f1184855bd78412238939419&scope.project.id=930b378be44b4bb7a2009c72a424f1d6&role.id=5ca112885467422684e134cb0f94865a => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:09.308134 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-9a24c76d-db85-4f6c-8ab8-5dc101545dc9 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:09.309583 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9a24c76d-db85-4f6c-8ab8-5dc101545dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:09.325151 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:09.405833 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 81/158] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:09.458167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061e503-7146-49d2-9afc-4ad3d1a44170 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:09.540111 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:09.607211 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061e503-7146-49d2-9afc-4ad3d1a44170 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:09.650504 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049200f-fc10-44ab-a1f9-760e230f8e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:09.651372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049200f-fc10-44ab-a1f9-760e230f8e1b None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:09.651670 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049200f-fc10-44ab-a1f9-760e230f8e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:09.651958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049200f-fc10-44ab-a1f9-760e230f8e1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:09.665363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049200f-fc10-44ab-a1f9-760e230f8e1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:09.677048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-8049200f-fc10-44ab-a1f9-760e230f8e1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:09.677048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:09.677048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:09.677048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:09.677048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:09.677048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:09.677048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:09.677048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:09.677048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:09.677048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:09.677048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:09.677048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:09.677048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:09.677048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:09.677048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:09.677048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:09.677987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:09.677987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:09.677987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:09.677987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:09.677987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:09.677987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:09.677987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:09.677987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:09.677987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:09.677987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:09.677987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:09.677987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:09.677987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:09.677987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:09.677987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:09.677987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:09.678837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:09.678837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:09.678837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:09.678837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:09.678837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:09.678837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:09.678837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:09.678837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:09.678837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:09.678837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:09.678837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:09.678837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:09.678837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 78/159] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:09.688103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24c76d-db85-4f6c-8ab8-5dc101545dc9 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:09.697005 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd832f4c-967a-4971-b9f8-964fa8a92e13 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:09.697295 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd832f4c-967a-4971-b9f8-964fa8a92e13 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:09.697577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd832f4c-967a-4971-b9f8-964fa8a92e13 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:09.699272 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 79/160] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:09.704966 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba838ad-6a7e-4ccf-aa41-1a033749756e None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:09.705260 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba838ad-6a7e-4ccf-aa41-1a033749756e None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:09.705547 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba838ad-6a7e-4ccf-aa41-1a033749756e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:09.728513 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-2ba838ad-6a7e-4ccf-aa41-1a033749756e None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:09.729761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-2ba838ad-6a7e-4ccf-aa41-1a033749756e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:09.804928 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 80/161] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:20:09.829424 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba838ad-6a7e-4ccf-aa41-1a033749756e None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:09.859978 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24c76d-db85-4f6c-8ab8-5dc101545dc9 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:09.883382 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:09.933381 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764e59c-3e84-4fc2-b605-4a238310909a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:09.934496 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764e59c-3e84-4fc2-b605-4a238310909a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:09.935668 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764e59c-3e84-4fc2-b605-4a238310909a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:09.936756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764e59c-3e84-4fc2-b605-4a238310909a None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:09.937746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7764e59c-3e84-4fc2-b605-4a238310909a None admin] RBAC: Adding query filter params (name=member) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:09.938259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7764e59c-3e84-4fc2-b605-4a238310909a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:09.940095 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7764e59c-3e84-4fc2-b605-4a238310909a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:09.945594 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba838ad-6a7e-4ccf-aa41-1a033749756e None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:09.954905 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 82/162] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:09.956584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ee7b7-5df0-4079-bd77-fce54789820a None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:09.957097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ee7b7-5df0-4079-bd77-fce54789820a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:09.957566 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ee7b7-5df0-4079-bd77-fce54789820a None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:09.959168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 83/163] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:09.964776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6596e93-d076-4570-9191-e77796c8f992 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:09.965367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6596e93-d076-4570-9191-e77796c8f992 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:09.965858 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6596e93-d076-4570-9191-e77796c8f992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:09.998344 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c6596e93-d076-4570-9191-e77796c8f992 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:09.999626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c6596e93-d076-4570-9191-e77796c8f992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:09.999919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b00dbd-e8d0-44ce-bad3-e28d925e6d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:10.000978 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b00dbd-e8d0-44ce-bad3-e28d925e6d25 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:10.001540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b00dbd-e8d0-44ce-bad3-e28d925e6d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:10.001970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:10.003308 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b00dbd-e8d0-44ce-bad3-e28d925e6d25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:10.023516 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b00dbd-e8d0-44ce-bad3-e28d925e6d25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:10.038449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-56b00dbd-e8d0-44ce-bad3-e28d925e6d25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:10.038449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:10.038449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:10.038449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:10.038449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:10.038449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:10.038449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:10.038449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:10.038449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:10.038449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:10.038449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:10.038449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:10.038449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:10.038449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:10.038449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:10.038449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:10.039415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:10.039415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:10.039415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:10.039415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:10.039415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:10.039415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:10.039415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:10.039415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:10.039415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:10.039415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:10.039415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:10.039415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:10.039415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:10.039415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:10.039415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:10.039415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:10.040198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:10.040198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:10.040198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:10.040198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:10.040198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:10.040198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:10.040198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:10.040198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:10.040198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:10.040198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:10.040198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:10.040198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:10.046286 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 81/164] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:20:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:10.048480 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b00dbd-e8d0-44ce-bad3-e28d925e6d25 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:10.127424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 84/165] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:10.129052 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6596e93-d076-4570-9191-e77796c8f992 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:10.195681 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b00dbd-e8d0-44ce-bad3-e28d925e6d25 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:10.196767 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:10.267665 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8d0fa-b94f-4b7e-b136-dd72c833dbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:10.268816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8d0fa-b94f-4b7e-b136-dd72c833dbc2 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:10.269102 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8d0fa-b94f-4b7e-b136-dd72c833dbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:10.269380 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8d0fa-b94f-4b7e-b136-dd72c833dbc2 None admin] PATH_INFO: `/v3/users/2f2bd63cb926471a9ba1d25571f4dd51` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:10.270200 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6596e93-d076-4570-9191-e77796c8f992 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:10.275758 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8d0fa-b94f-4b7e-b136-dd72c833dbc2 None admin] RBAC: Authorizing `identity:get_user(user_id=2f2bd63cb926471a9ba1d25571f4dd51)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:10.277333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8d0fa-b94f-4b7e-b136-dd72c833dbc2 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:10.279226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 82/166] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:10 2022] GET /identity/v3/users/2f2bd63cb926471a9ba1d25571f4dd51 => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:10.280431 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8d0fa-b94f-4b7e-b136-dd72c833dbc2 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:10.312501 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e1ca5-4b7f-4536-9a9c-f03571cce05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:10.313574 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e1ca5-4b7f-4536-9a9c-f03571cce05f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:10.314086 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e1ca5-4b7f-4536-9a9c-f03571cce05f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:10.314605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e1ca5-4b7f-4536-9a9c-f03571cce05f None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:10.315577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e1ca5-4b7f-4536-9a9c-f03571cce05f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:10.316643 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e1ca5-4b7f-4536-9a9c-f03571cce05f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:10.318672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e1ca5-4b7f-4536-9a9c-f03571cce05f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:10.324751 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:10.332078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 85/167] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:20:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:10.333162 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e1ca5-4b7f-4536-9a9c-f03571cce05f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:10.452812 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8d0fa-b94f-4b7e-b136-dd72c833dbc2 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:10.495266 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cfa9f-2225-4e95-a5de-d6c7cebf570e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:10.496596 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cfa9f-2225-4e95-a5de-d6c7cebf570e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:10.496986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cfa9f-2225-4e95-a5de-d6c7cebf570e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:10.497382 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cfa9f-2225-4e95-a5de-d6c7cebf570e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:10.500945 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e1ca5-4b7f-4536-9a9c-f03571cce05f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:10.511983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cfa9f-2225-4e95-a5de-d6c7cebf570e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:10.521776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-1c0cfa9f-2225-4e95-a5de-d6c7cebf570e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:10.521776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:10.521776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:10.521776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:10.521776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:10.521776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:10.521776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:10.521776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:10.521776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:10.521776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:10.521776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:10.521776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:10.521776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:10.521776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:10.521776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:10.521776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:10.522894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:10.522894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:10.522894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:10.522894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:10.522894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:10.522894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:10.522894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:10.522894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:10.522894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:10.522894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:10.522894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:10.522894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:10.522894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:10.522894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:10.522894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:10.522894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:10.523802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:10.523802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:10.523802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:10.523802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:10.523802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:10.523802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:10.523802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:10.523802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:10.523802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:10.523802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:10.523802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:10.523802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:10.524418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 83/168] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:10.525314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cfa9f-2225-4e95-a5de-d6c7cebf570e None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:10.568192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:10.575416 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52495a4b-6add-41fe-a6ca-ee927a4d4e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:10.576422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52495a4b-6add-41fe-a6ca-ee927a4d4e0e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:10.576939 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52495a4b-6add-41fe-a6ca-ee927a4d4e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:10.577436 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52495a4b-6add-41fe-a6ca-ee927a4d4e0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:10.745300 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cfa9f-2225-4e95-a5de-d6c7cebf570e None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:10.751518 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:10.775850 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52495a4b-6add-41fe-a6ca-ee927a4d4e0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:10.777682 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52495a4b-6add-41fe-a6ca-ee927a4d4e0e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:10.830157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb921b05-4e3a-44d4-af99-a5f8dce573f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:10.831082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb921b05-4e3a-44d4-af99-a5f8dce573f3 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:10.834958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb921b05-4e3a-44d4-af99-a5f8dce573f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:10.834958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb921b05-4e3a-44d4-af99-a5f8dce573f3 None admin] PATH_INFO: `/v3/users/2f2bd63cb926471a9ba1d25571f4dd51` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:10.834958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb921b05-4e3a-44d4-af99-a5f8dce573f3 None admin] RBAC: Authorizing `identity:get_user(user_id=2f2bd63cb926471a9ba1d25571f4dd51)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:10.834958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb921b05-4e3a-44d4-af99-a5f8dce573f3 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:10.840210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 84/169] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:10 2022] GET /identity/v3/users/2f2bd63cb926471a9ba1d25571f4dd51 => generated 275 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:10.841006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb921b05-4e3a-44d4-af99-a5f8dce573f3 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:10.972633 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 86/170] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:10.973520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52495a4b-6add-41fe-a6ca-ee927a4d4e0e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:11.047395 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:11.051954 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb921b05-4e3a-44d4-af99-a5f8dce573f3 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:11.124797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fede5-060c-4c20-978b-dc6fefac7440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:11.125771 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fede5-060c-4c20-978b-dc6fefac7440 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:11.126193 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fede5-060c-4c20-978b-dc6fefac7440 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:11.126604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fede5-060c-4c20-978b-dc6fefac7440 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:11.127754 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fede5-060c-4c20-978b-dc6fefac7440 None admin] RBAC: Adding query filter params (name=member) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:11.128225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fede5-060c-4c20-978b-dc6fefac7440 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:11.129854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fede5-060c-4c20-978b-dc6fefac7440 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:11.130816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:11.138960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52495a4b-6add-41fe-a6ca-ee927a4d4e0e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:11.146716 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 85/171] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:11.151367 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fede5-060c-4c20-978b-dc6fefac7440 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:11.173803 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89634679-9806-4089-86df-09606a47b6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:11.174448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89634679-9806-4089-86df-09606a47b6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:11.174733 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89634679-9806-4089-86df-09606a47b6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:11.175038 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89634679-9806-4089-86df-09606a47b6e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:11.249439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89634679-9806-4089-86df-09606a47b6e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:11.250969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89634679-9806-4089-86df-09606a47b6e9 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:11.353057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fede5-060c-4c20-978b-dc6fefac7440 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:11.357131 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:11.362476 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 87/172] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:11.363346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89634679-9806-4089-86df-09606a47b6e9 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:11.432577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb76813-1d36-4155-b438-115cb9c4a78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:11.433859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb76813-1d36-4155-b438-115cb9c4a78c None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:11.434372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb76813-1d36-4155-b438-115cb9c4a78c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:11.434847 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb76813-1d36-4155-b438-115cb9c4a78c None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:11.436854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb76813-1d36-4155-b438-115cb9c4a78c None admin] RBAC: Authorizing `identity:get_project(project_id=930b378be44b4bb7a2009c72a424f1d6)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:11.438561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb76813-1d36-4155-b438-115cb9c4a78c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:11.446963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 86/173] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:11 2022] GET /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6 => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:11.448502 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb76813-1d36-4155-b438-115cb9c4a78c None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:11.471296 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89634679-9806-4089-86df-09606a47b6e9 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:11.503933 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:11.514960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db367c0a-9abf-4f60-a639-f726d967fa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:11.514960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db367c0a-9abf-4f60-a639-f726d967fa20 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:11.514960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db367c0a-9abf-4f60-a639-f726d967fa20 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:11.514960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db367c0a-9abf-4f60-a639-f726d967fa20 None admin] PATH_INFO: `/v3/users/2f2bd63cb926471a9ba1d25571f4dd51` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:11.515359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db367c0a-9abf-4f60-a639-f726d967fa20 None admin] RBAC: Authorizing `identity:get_user(user_id=2f2bd63cb926471a9ba1d25571f4dd51)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:11.515359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db367c0a-9abf-4f60-a639-f726d967fa20 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:11.516821 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 88/174] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:11 2022] GET /identity/v3/users/2f2bd63cb926471a9ba1d25571f4dd51 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:11.517656 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db367c0a-9abf-4f60-a639-f726d967fa20 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:11.592026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:11.631752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db367c0a-9abf-4f60-a639-f726d967fa20 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:11.674828 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7105d-4e02-48e6-8e32-9152f1ef127e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:11.675645 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7105d-4e02-48e6-8e32-9152f1ef127e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:11.675957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7105d-4e02-48e6-8e32-9152f1ef127e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:11.676273 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7105d-4e02-48e6-8e32-9152f1ef127e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:11.677182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7105d-4e02-48e6-8e32-9152f1ef127e None admin] RBAC: Adding query filter params (role.id=b778e530d54948c7bc31f5b9231ea288, scope.project.id=930b378be44b4bb7a2009c72a424f1d6, user.id=2f2bd63cb926471a9ba1d25571f4dd51) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:11.677591 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7105d-4e02-48e6-8e32-9152f1ef127e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:11.679552 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7105d-4e02-48e6-8e32-9152f1ef127e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:11.689639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 89/175] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:11 2022] GET /identity/v3/role_assignments?user.id=2f2bd63cb926471a9ba1d25571f4dd51&scope.project.id=930b378be44b4bb7a2009c72a424f1d6&role.id=b778e530d54948c7bc31f5b9231ea288 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:11.691045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6db3d0-710c-4ad2-a236-137859a1df66 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:11.691338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6db3d0-710c-4ad2-a236-137859a1df66 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:11.691758 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6db3d0-710c-4ad2-a236-137859a1df66 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:11.695308 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 90/176] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:11.696576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6db3d0-710c-4ad2-a236-137859a1df66 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:11.705554 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb76813-1d36-4155-b438-115cb9c4a78c None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:11.787055 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a656e-aae7-483b-9983-6b0bd9e7bde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:11.788549 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a656e-aae7-483b-9983-6b0bd9e7bde2 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:11.789158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a656e-aae7-483b-9983-6b0bd9e7bde2 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:11.789634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a656e-aae7-483b-9983-6b0bd9e7bde2 None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:11.791312 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a656e-aae7-483b-9983-6b0bd9e7bde2 None admin] RBAC: Authorizing `identity:get_project(project_id=930b378be44b4bb7a2009c72a424f1d6)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:11.793252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a656e-aae7-483b-9983-6b0bd9e7bde2 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:11.796669 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 87/177] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:11 2022] GET /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6 => generated 298 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:11.798552 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b674d2e-120f-4235-89dd-76b550d9ccea None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:11.800604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b674d2e-120f-4235-89dd-76b550d9ccea None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:11.801197 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b674d2e-120f-4235-89dd-76b550d9ccea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:11.822379 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-4b674d2e-120f-4235-89dd-76b550d9ccea None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:11.824061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4b674d2e-120f-4235-89dd-76b550d9ccea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:11.844673 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6db3d0-710c-4ad2-a236-137859a1df66 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:11.885090 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:11.892972 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f348313-832e-4491-84de-4df6806a313e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:11.894098 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f348313-832e-4491-84de-4df6806a313e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:11.894382 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f348313-832e-4491-84de-4df6806a313e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:11.894658 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f348313-832e-4491-84de-4df6806a313e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:11.930418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 88/178] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:11.932456 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba79c8-ccbb-4d7d-b32a-3e6bc02df323 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:11.932960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba79c8-ccbb-4d7d-b32a-3e6bc02df323 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:11.933449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba79c8-ccbb-4d7d-b32a-3e6bc02df323 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:11.941397 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 89/179] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:11 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:11.942429 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27003802-22df-415e-b2d8-b5c228506736 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:11.942715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27003802-22df-415e-b2d8-b5c228506736 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:11.943022 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27003802-22df-415e-b2d8-b5c228506736 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:11.944675 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 90/180] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:11.945065 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27003802-22df-415e-b2d8-b5c228506736 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:11.982022 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:11.991768 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f348313-832e-4491-84de-4df6806a313e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:11.993273 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f348313-832e-4491-84de-4df6806a313e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:12.073334 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27003802-22df-415e-b2d8-b5c228506736 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:12.097609 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 91/181] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:12.099013 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb515d7-7c99-4b84-939b-ec1f9e9c33b7 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:12.099266 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb515d7-7c99-4b84-939b-ec1f9e9c33b7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:12.099615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb515d7-7c99-4b84-939b-ec1f9e9c33b7 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:12.101079 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 92/182] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:12.104383 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbdd99-59c7-4dc8-92e8-98ae7f681a86 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:12.104685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbdd99-59c7-4dc8-92e8-98ae7f681a86 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:12.105307 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbdd99-59c7-4dc8-92e8-98ae7f681a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:12.111383 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ded297-0beb-4613-82c4-c34275439656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:12.112055 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ded297-0beb-4613-82c4-c34275439656 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:12.112341 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ded297-0beb-4613-82c4-c34275439656 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:12.112619 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ded297-0beb-4613-82c4-c34275439656 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:12.113463 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ded297-0beb-4613-82c4-c34275439656 None admin] RBAC: Adding query filter params (role.id=6b2eb3e4cefd4a2db4f18e04a158600a, scope.project.id=930b378be44b4bb7a2009c72a424f1d6, user.id=2f2bd63cb926471a9ba1d25571f4dd51) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:12.113777 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ded297-0beb-4613-82c4-c34275439656 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:12.115289 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ded297-0beb-4613-82c4-c34275439656 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:12.121496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 91/183] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:11 2022] GET /identity/v3/role_assignments?user.id=2f2bd63cb926471a9ba1d25571f4dd51&scope.project.id=930b378be44b4bb7a2009c72a424f1d6&role.id=6b2eb3e4cefd4a2db4f18e04a158600a => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:12.122622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f2cbdd99-59c7-4dc8-92e8-98ae7f681a86 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:12.122740 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42b712-b261-40ac-a53d-0307c7fe5bc4 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:12.123036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42b712-b261-40ac-a53d-0307c7fe5bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:12.123317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42b712-b261-40ac-a53d-0307c7fe5bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:12.123991 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f2cbdd99-59c7-4dc8-92e8-98ae7f681a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:12.150115 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-3b42b712-b261-40ac-a53d-0307c7fe5bc4 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:12.151317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3b42b712-b261-40ac-a53d-0307c7fe5bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:12.159840 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:12.218838 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 93/184] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:12.219922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbdd99-59c7-4dc8-92e8-98ae7f681a86 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:12.242875 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 92/185] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:12.244436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42b712-b261-40ac-a53d-0307c7fe5bc4 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:12.273972 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:12.370126 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42b712-b261-40ac-a53d-0307c7fe5bc4 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:12.376744 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbdd99-59c7-4dc8-92e8-98ae7f681a86 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:12.404468 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e99d9-7480-406f-a545-13588cbecbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:12.405830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e99d9-7480-406f-a545-13588cbecbb0 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:12.406111 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e99d9-7480-406f-a545-13588cbecbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:12.406381 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e99d9-7480-406f-a545-13588cbecbb0 None admin] PATH_INFO: `/v3/projects/05a1fc92d37549878f650ecbc4773f33` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:12.407601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e99d9-7480-406f-a545-13588cbecbb0 None admin] RBAC: Authorizing `identity:get_project(project_id=05a1fc92d37549878f650ecbc4773f33)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:12.409041 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e99d9-7480-406f-a545-13588cbecbb0 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:12.410792 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 93/186] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:12 2022] GET /identity/v3/projects/05a1fc92d37549878f650ecbc4773f33 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:12.412061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a395d-8fe8-4ddc-b6ea-0526fe75e9cf None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:12.412334 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a395d-8fe8-4ddc-b6ea-0526fe75e9cf None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:12.412607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a395d-8fe8-4ddc-b6ea-0526fe75e9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:12.417483 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497cb97d-a719-4ce2-9f6c-908802f869dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:12.418291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cb97d-a719-4ce2-9f6c-908802f869dc None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:12.418577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cb97d-a719-4ce2-9f6c-908802f869dc None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:12.418853 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cb97d-a719-4ce2-9f6c-908802f869dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:12.430117 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-637a395d-8fe8-4ddc-b6ea-0526fe75e9cf None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:12.430485 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497cb97d-a719-4ce2-9f6c-908802f869dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:12.431484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-637a395d-8fe8-4ddc-b6ea-0526fe75e9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:12.438304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-497cb97d-a719-4ce2-9f6c-908802f869dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:20:12.438304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:12.438304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:12.438304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:12.438304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:12.438304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:12.438304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:12.438304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:12.438304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:12.438304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:12.438304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:12.438304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:12.438304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:12.438304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:12.438304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:12.438304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:12.439217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:12.439217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:12.439217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:12.439217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:12.439217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:12.439217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:12.439217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:12.439217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:12.439217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:12.439217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:12.439217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:12.439217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:12.439217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:12.439217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:12.439217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:12.439217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:12.440329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:12.440329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:12.440329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:12.440329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:12.440329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:12.440329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:12.440329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:12.440329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:12.440329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:12.440329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:12.440329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:20:12.440329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:12.440329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 94/187] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:20:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:12.441200 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497cb97d-a719-4ce2-9f6c-908802f869dc None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:12.466556 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:12.511913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 94/188] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:12.515921 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a395d-8fe8-4ddc-b6ea-0526fe75e9cf None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:12.538130 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:12.554994 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497cb97d-a719-4ce2-9f6c-908802f869dc None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:12.591849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d7504-7162-40d2-acf8-e7fcdbf013b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:12.593532 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d7504-7162-40d2-acf8-e7fcdbf013b2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:12.593812 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d7504-7162-40d2-acf8-e7fcdbf013b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:12.594083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d7504-7162-40d2-acf8-e7fcdbf013b2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:12.605648 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d7504-7162-40d2-acf8-e7fcdbf013b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:12.613516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-459d7504-7162-40d2-acf8-e7fcdbf013b2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 20:20:12.613516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:12.613516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:12.613516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:12.613516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:12.613516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:12.613516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:12.613516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:12.613516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:12.613516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:12.613516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:12.613516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:12.613516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:12.613516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:12.613516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:12.613516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:12.614453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:12.614453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:12.614453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:12.614453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:12.614453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:12.614453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:12.614453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:12.614453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:12.614453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:12.614453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:12.614453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:12.614453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:12.614453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:12.614453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:12.614453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:12.614453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:12.615297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:12.615297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:12.615297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:12.615297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:12.615297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:12.615297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:12.615297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:12.615297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:12.615297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:12.615297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:12.615297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 20:20:12.615297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:12.615297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 95/189] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:12.615946 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d7504-7162-40d2-acf8-e7fcdbf013b2 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:12.633392 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a395d-8fe8-4ddc-b6ea-0526fe75e9cf None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:12.669248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8cc96-aa67-4f82-8eae-f194df172c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:12.669974 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8cc96-aa67-4f82-8eae-f194df172c19 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:12.670254 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8cc96-aa67-4f82-8eae-f194df172c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:12.670572 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8cc96-aa67-4f82-8eae-f194df172c19 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:12.681229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8cc96-aa67-4f82-8eae-f194df172c19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:12.689267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-1eb8cc96-aa67-4f82-8eae-f194df172c19 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:12.689267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:12.689267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:12.689267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:12.689267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:12.689267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:12.689267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:12.689267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:12.689267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:12.689267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:12.689267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:12.689267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:12.689267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:12.689267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:12.689267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:12.689267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:12.690174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:12.690174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:12.690174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:12.690174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:12.690174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:12.690174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:12.690174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:12.690174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:12.690174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:12.690174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:12.690174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:12.690174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:12.690174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:12.690174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:12.690174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:12.690174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:12.690985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:12.690985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:12.690985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:12.690985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:12.690985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:12.690985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:12.690985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:12.690985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:12.690985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:12.690985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:12.690985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:12.690985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:12.690985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 95/190] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:20:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:12.691644 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8cc96-aa67-4f82-8eae-f194df172c19 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:12.710829 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:12.731059 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d7504-7162-40d2-acf8-e7fcdbf013b2 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:12.764075 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c527c-3ee0-4edf-a2d9-5dcc6d37728e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:12.765238 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c527c-3ee0-4edf-a2d9-5dcc6d37728e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:12.765521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c527c-3ee0-4edf-a2d9-5dcc6d37728e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:12.765799 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c527c-3ee0-4edf-a2d9-5dcc6d37728e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:12.766634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c527c-3ee0-4edf-a2d9-5dcc6d37728e None admin] RBAC: Adding query filter params (role.id=b778e530d54948c7bc31f5b9231ea288, scope.project.id=05a1fc92d37549878f650ecbc4773f33, user.id=2f2bd63cb926471a9ba1d25571f4dd51) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:12.766968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c527c-3ee0-4edf-a2d9-5dcc6d37728e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:12.768689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c527c-3ee0-4edf-a2d9-5dcc6d37728e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:12.774740 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 96/191] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:12 2022] GET /identity/v3/role_assignments?user.id=2f2bd63cb926471a9ba1d25571f4dd51&scope.project.id=05a1fc92d37549878f650ecbc4773f33&role.id=b778e530d54948c7bc31f5b9231ea288 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:12.775587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c527c-3ee0-4edf-a2d9-5dcc6d37728e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:12.781077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:12.804000 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8cc96-aa67-4f82-8eae-f194df172c19 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:12.843092 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f1069-7d7f-443f-aa07-7d5c8d6fbc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:12.843785 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f1069-7d7f-443f-aa07-7d5c8d6fbc65 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:12.844077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f1069-7d7f-443f-aa07-7d5c8d6fbc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:12.844357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f1069-7d7f-443f-aa07-7d5c8d6fbc65 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:12.845133 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f1069-7d7f-443f-aa07-7d5c8d6fbc65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:12.845442 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f1069-7d7f-443f-aa07-7d5c8d6fbc65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:12.846874 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f1069-7d7f-443f-aa07-7d5c8d6fbc65 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:12.857207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 96/192] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:12.858101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f1069-7d7f-443f-aa07-7d5c8d6fbc65 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:12.918987 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c527c-3ee0-4edf-a2d9-5dcc6d37728e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:12.963972 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea4df4-991a-4904-a10e-bbc0374520ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:12.965128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea4df4-991a-4904-a10e-bbc0374520ea None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:12.965530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea4df4-991a-4904-a10e-bbc0374520ea None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:12.966038 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea4df4-991a-4904-a10e-bbc0374520ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:12.967366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea4df4-991a-4904-a10e-bbc0374520ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:12.969032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea4df4-991a-4904-a10e-bbc0374520ea None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:12.970777 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 97/193] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:12.971284 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea4df4-991a-4904-a10e-bbc0374520ea None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:12.987002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f1069-7d7f-443f-aa07-7d5c8d6fbc65 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:12.994259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:13.028161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcc95c-8f53-46d2-8059-c67aa7942b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:13.028846 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcc95c-8f53-46d2-8059-c67aa7942b26 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:13.029131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcc95c-8f53-46d2-8059-c67aa7942b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:13.029406 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcc95c-8f53-46d2-8059-c67aa7942b26 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:13.030191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcc95c-8f53-46d2-8059-c67aa7942b26 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:13.030501 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcc95c-8f53-46d2-8059-c67aa7942b26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:13.031930 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcc95c-8f53-46d2-8059-c67aa7942b26 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:13.041082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 97/194] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:13.042206 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcc95c-8f53-46d2-8059-c67aa7942b26 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:13.045781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:13.088740 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea4df4-991a-4904-a10e-bbc0374520ea None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:13.123288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318df09f-0cd4-4759-a7a5-a988f64a982c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:13.124478 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318df09f-0cd4-4759-a7a5-a988f64a982c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:13.124770 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318df09f-0cd4-4759-a7a5-a988f64a982c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:13.125041 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318df09f-0cd4-4759-a7a5-a988f64a982c None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:13.125831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318df09f-0cd4-4759-a7a5-a988f64a982c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:13.126142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318df09f-0cd4-4759-a7a5-a988f64a982c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:13.127577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318df09f-0cd4-4759-a7a5-a988f64a982c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:13.135807 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 98/195] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:20:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:13.136604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318df09f-0cd4-4759-a7a5-a988f64a982c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:13.149457 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcc95c-8f53-46d2-8059-c67aa7942b26 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:13.183531 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75c9a8-385d-42f9-95dc-a0153fe705d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:13.184481 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75c9a8-385d-42f9-95dc-a0153fe705d7 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:13.184767 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75c9a8-385d-42f9-95dc-a0153fe705d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:13.185045 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75c9a8-385d-42f9-95dc-a0153fe705d7 None admin] PATH_INFO: `/v3/users/c420c0e76add44229d13aeb769ce9476` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:13.201033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75c9a8-385d-42f9-95dc-a0153fe705d7 None admin] RBAC: Authorizing `identity:get_user(user_id=c420c0e76add44229d13aeb769ce9476)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:13.202509 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75c9a8-385d-42f9-95dc-a0153fe705d7 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:13.204315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 98/196] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:13 2022] GET /identity/v3/users/c420c0e76add44229d13aeb769ce9476 => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:13.205089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75c9a8-385d-42f9-95dc-a0153fe705d7 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:13.217786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:13.242296 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318df09f-0cd4-4759-a7a5-a988f64a982c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:13.265840 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:13.280238 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df68d1-0a1e-40d9-b0e1-87f5e83ce32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:13.281172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df68d1-0a1e-40d9-b0e1-87f5e83ce32f None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:13.281489 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df68d1-0a1e-40d9-b0e1-87f5e83ce32f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:13.281733 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df68d1-0a1e-40d9-b0e1-87f5e83ce32f None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:13.282495 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df68d1-0a1e-40d9-b0e1-87f5e83ce32f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:13.283972 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df68d1-0a1e-40d9-b0e1-87f5e83ce32f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:13.310880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 99/197] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 14 20:20:13 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:20:13.312314 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df68d1-0a1e-40d9-b0e1-87f5e83ce32f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:13.315525 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75c9a8-385d-42f9-95dc-a0153fe705d7 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:13.360299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b2ac4-3e05-4ca4-9070-c2f41cea9b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:13.361223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b2ac4-3e05-4ca4-9070-c2f41cea9b54 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:13.361500 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b2ac4-3e05-4ca4-9070-c2f41cea9b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:13.361818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b2ac4-3e05-4ca4-9070-c2f41cea9b54 None admin] PATH_INFO: `/v3/users/700a9c53c7624ddc839cbd2a4c2c169f` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:13.363349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b2ac4-3e05-4ca4-9070-c2f41cea9b54 None admin] RBAC: Authorizing `identity:get_user(user_id=700a9c53c7624ddc839cbd2a4c2c169f)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:13.364954 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b2ac4-3e05-4ca4-9070-c2f41cea9b54 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:13.366889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 99/198] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:13 2022] GET /identity/v3/users/700a9c53c7624ddc839cbd2a4c2c169f => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:13.367634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b2ac4-3e05-4ca4-9070-c2f41cea9b54 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:13.435218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df68d1-0a1e-40d9-b0e1-87f5e83ce32f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:13.473187 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:13.480622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75d4cb-41fd-4d43-b06e-9115ec2d00c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:13.481764 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75d4cb-41fd-4d43-b06e-9115ec2d00c2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:13.482202 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75d4cb-41fd-4d43-b06e-9115ec2d00c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:13.482480 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75d4cb-41fd-4d43-b06e-9115ec2d00c2 None admin] PATH_INFO: `/v3/users/c420c0e76add44229d13aeb769ce9476` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:13.484275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b2ac4-3e05-4ca4-9070-c2f41cea9b54 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:13.484737 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75d4cb-41fd-4d43-b06e-9115ec2d00c2 None admin] RBAC: Authorizing `identity:get_user(user_id=c420c0e76add44229d13aeb769ce9476)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:13.485828 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75d4cb-41fd-4d43-b06e-9115ec2d00c2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:13.488861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 100/199] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:13 2022] GET /identity/v3/users/c420c0e76add44229d13aeb769ce9476 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:13.488861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75d4cb-41fd-4d43-b06e-9115ec2d00c2 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:13.520068 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:13.536330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdaca1a-938d-4d18-a58d-df830b689943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:13.537007 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdaca1a-938d-4d18-a58d-df830b689943 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:13.537294 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdaca1a-938d-4d18-a58d-df830b689943 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:13.537574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdaca1a-938d-4d18-a58d-df830b689943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:13.621740 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75d4cb-41fd-4d43-b06e-9115ec2d00c2 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:13.630501 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdaca1a-938d-4d18-a58d-df830b689943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:13.634493 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdaca1a-938d-4d18-a58d-df830b689943 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:13.668112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304a219-b175-43c0-a84b-bc06064f43aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:13.668764 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304a219-b175-43c0-a84b-bc06064f43aa None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:13.669046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304a219-b175-43c0-a84b-bc06064f43aa None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:13.669314 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304a219-b175-43c0-a84b-bc06064f43aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:13.744674 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 100/200] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:13.745936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdaca1a-938d-4d18-a58d-df830b689943 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:13.754603 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:13.757774 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304a219-b175-43c0-a84b-bc06064f43aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:13.760643 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304a219-b175-43c0-a84b-bc06064f43aa None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:13.781042 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:13.857385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdaca1a-938d-4d18-a58d-df830b689943 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:13.877366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 101/201] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:13.878434 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304a219-b175-43c0-a84b-bc06064f43aa None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:13.893700 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce22983-1919-451c-b931-6beb54e29c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:13.894520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce22983-1919-451c-b931-6beb54e29c9e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:13.894902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce22983-1919-451c-b931-6beb54e29c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:13.895294 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce22983-1919-451c-b931-6beb54e29c9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:13.968311 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce22983-1919-451c-b931-6beb54e29c9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:13.969938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce22983-1919-451c-b931-6beb54e29c9e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:13.988410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:13.992442 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304a219-b175-43c0-a84b-bc06064f43aa None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:13.999522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:14.031542 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb360c-7369-45b7-be11-728cd7811a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:14.032687 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb360c-7369-45b7-be11-728cd7811a8e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:14.033112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb360c-7369-45b7-be11-728cd7811a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:14.033520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb360c-7369-45b7-be11-728cd7811a8e None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:14.035096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb360c-7369-45b7-be11-728cd7811a8e None admin] RBAC: Authorizing `identity:get_project(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:14.036743 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb360c-7369-45b7-be11-728cd7811a8e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:14.038601 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 102/202] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:13 2022] GET /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:14.039658 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb360c-7369-45b7-be11-728cd7811a8e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:14.086917 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 101/203] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:14.088139 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce22983-1919-451c-b931-6beb54e29c9e None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:14.190676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb360c-7369-45b7-be11-728cd7811a8e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:14.198554 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce22983-1919-451c-b931-6beb54e29c9e None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:14.230877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f97d55-1fd1-434b-a09e-c9825d77c6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:14.232484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f97d55-1fd1-434b-a09e-c9825d77c6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:14.233594 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f97d55-1fd1-434b-a09e-c9825d77c6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:14.234089 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f97d55-1fd1-434b-a09e-c9825d77c6f4 None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:14.235132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:14.236360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f97d55-1fd1-434b-a09e-c9825d77c6f4 None admin] RBAC: Authorizing `identity:get_project(project_id=930b378be44b4bb7a2009c72a424f1d6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:14.238176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f97d55-1fd1-434b-a09e-c9825d77c6f4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:14.240646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 103/204] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:14 2022] GET /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:14.241795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f97d55-1fd1-434b-a09e-c9825d77c6f4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:14.244253 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c716dc8-5f98-4486-a35d-6da7fce8c50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:14.245260 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c716dc8-5f98-4486-a35d-6da7fce8c50e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:14.245573 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c716dc8-5f98-4486-a35d-6da7fce8c50e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:14.245853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c716dc8-5f98-4486-a35d-6da7fce8c50e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:14.247729 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c716dc8-5f98-4486-a35d-6da7fce8c50e None admin] RBAC: Adding query filter params (role.id=5ca112885467422684e134cb0f94865a, scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5, user.id=c420c0e76add44229d13aeb769ce9476) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:14.248160 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c716dc8-5f98-4486-a35d-6da7fce8c50e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:14.250258 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c716dc8-5f98-4486-a35d-6da7fce8c50e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:14.258985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 102/205] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:14 2022] GET /identity/v3/role_assignments?user.id=c420c0e76add44229d13aeb769ce9476&scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5&role.id=5ca112885467422684e134cb0f94865a => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:14.259765 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c716dc8-5f98-4486-a35d-6da7fce8c50e None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:14.287369 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:14.417959 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f97d55-1fd1-434b-a09e-c9825d77c6f4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:14.418535 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c716dc8-5f98-4486-a35d-6da7fce8c50e None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:14.464388 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196eb05-3307-454d-b11a-8e7a4ce0f1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:14.465103 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196eb05-3307-454d-b11a-8e7a4ce0f1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:14.465349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196eb05-3307-454d-b11a-8e7a4ce0f1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:14.465634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196eb05-3307-454d-b11a-8e7a4ce0f1e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:14.466479 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196eb05-3307-454d-b11a-8e7a4ce0f1e7 None admin] RBAC: Adding query filter params (role.id=a7d155dd59914d22a32012302fa6d2d2, scope.project.id=930b378be44b4bb7a2009c72a424f1d6, user.id=700a9c53c7624ddc839cbd2a4c2c169f) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:14.466790 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196eb05-3307-454d-b11a-8e7a4ce0f1e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:14.475463 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196eb05-3307-454d-b11a-8e7a4ce0f1e7 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:14.488074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 103/206] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:14 2022] GET /identity/v3/role_assignments?user.id=700a9c53c7624ddc839cbd2a4c2c169f&scope.project.id=930b378be44b4bb7a2009c72a424f1d6&role.id=a7d155dd59914d22a32012302fa6d2d2 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:14.494598 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ede0e-e323-4a7b-bbe0-7b9fa955361a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:14.495561 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ede0e-e323-4a7b-bbe0-7b9fa955361a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:14.495851 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ede0e-e323-4a7b-bbe0-7b9fa955361a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:14.496137 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ede0e-e323-4a7b-bbe0-7b9fa955361a None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:14.497500 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ede0e-e323-4a7b-bbe0-7b9fa955361a None admin] RBAC: Authorizing `identity:get_project(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:14.499343 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ede0e-e323-4a7b-bbe0-7b9fa955361a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:14.501603 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 104/207] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:14 2022] GET /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:14.507729 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196eb05-3307-454d-b11a-8e7a4ce0f1e7 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:14.583337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:14.624002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196eb05-3307-454d-b11a-8e7a4ce0f1e7 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:14.663164 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc9226-8087-4f86-8732-832e6cc31da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:14.663852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc9226-8087-4f86-8732-832e6cc31da3 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:14.664138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc9226-8087-4f86-8732-832e6cc31da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:14.664420 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc9226-8087-4f86-8732-832e6cc31da3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:14.665292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc9226-8087-4f86-8732-832e6cc31da3 None admin] RBAC: Adding query filter params (role.id=6b2eb3e4cefd4a2db4f18e04a158600a, scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5, user.id=c420c0e76add44229d13aeb769ce9476) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:14.665611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc9226-8087-4f86-8732-832e6cc31da3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:14.667171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc9226-8087-4f86-8732-832e6cc31da3 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:14.674532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 104/208] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:14 2022] GET /identity/v3/role_assignments?user.id=c420c0e76add44229d13aeb769ce9476&scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5&role.id=6b2eb3e4cefd4a2db4f18e04a158600a => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:15.630595 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6943-d3f0-41b0-8e00-caf3adf2a90a None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:15.631133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6943-d3f0-41b0-8e00-caf3adf2a90a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:15.631206 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6943-d3f0-41b0-8e00-caf3adf2a90a None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:15.632866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 105/209] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:15.638855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531d8ec-2e67-490b-af63-5575dfa7927f None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:15.639461 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531d8ec-2e67-490b-af63-5575dfa7927f None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:15.639461 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531d8ec-2e67-490b-af63-5575dfa7927f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:15.657683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-e531d8ec-2e67-490b-af63-5575dfa7927f None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:15.658971 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e531d8ec-2e67-490b-af63-5575dfa7927f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:15.732084 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 105/210] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:15.752166 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f843396-9b9d-4feb-b833-878d93f536b1 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:15.752770 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f843396-9b9d-4feb-b833-878d93f536b1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:15.753239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f843396-9b9d-4feb-b833-878d93f536b1 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:15.754793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 106/211] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:15.757863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531d8ec-2e67-490b-af63-5575dfa7927f None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:15.760513 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a4258-a7cb-4b09-8561-c192775d3eae None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:15.761031 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a4258-a7cb-4b09-8561-c192775d3eae None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:15.761499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a4258-a7cb-4b09-8561-c192775d3eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:15.779380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-6c4a4258-a7cb-4b09-8561-c192775d3eae None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:15.781065 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6c4a4258-a7cb-4b09-8561-c192775d3eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:15.818856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:15.863429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 107/212] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:15.886983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531d8ec-2e67-490b-af63-5575dfa7927f None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:15.888298 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a4258-a7cb-4b09-8561-c192775d3eae None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:15.926921 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:15.936398 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e804c3-2d96-44fe-aaa6-2a2478b16e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:15.937681 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e804c3-2d96-44fe-aaa6-2a2478b16e34 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:15.937966 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e804c3-2d96-44fe-aaa6-2a2478b16e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:15.938241 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e804c3-2d96-44fe-aaa6-2a2478b16e34 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:15.954479 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e804c3-2d96-44fe-aaa6-2a2478b16e34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:15.965401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-43e804c3-2d96-44fe-aaa6-2a2478b16e34 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:15.965401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:15.965401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:15.965401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:15.965401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:15.965401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:15.965401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:15.965401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:15.965401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:15.965401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:15.965401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:15.965401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:15.965401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:15.965401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:15.965401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:15.965401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:15.966339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:15.966339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:15.966339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:15.966339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:15.966339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:15.966339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:15.966339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:15.966339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:15.966339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:15.966339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:15.966339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:15.966339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:15.966339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:15.966339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:15.966339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:15.966339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:15.967159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:15.967159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:15.967159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:15.967159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:15.967159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:15.967159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:15.967159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:15.967159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:15.967159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:15.967159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:15.967159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:15.967159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:15.967159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 106/213] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:15.972934 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e804c3-2d96-44fe-aaa6-2a2478b16e34 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:16.007810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a4258-a7cb-4b09-8561-c192775d3eae None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:16.045977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e00b5-6232-4415-91f8-4fca4a46f4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:16.047208 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e00b5-6232-4415-91f8-4fca4a46f4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:16.047686 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e00b5-6232-4415-91f8-4fca4a46f4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:16.048150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e00b5-6232-4415-91f8-4fca4a46f4e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:16.050144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59e00b5-6232-4415-91f8-4fca4a46f4e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:16.051897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59e00b5-6232-4415-91f8-4fca4a46f4e5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:16.058664 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 108/214] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:16.059776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e00b5-6232-4415-91f8-4fca4a46f4e5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:16.080697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e804c3-2d96-44fe-aaa6-2a2478b16e34 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:16.091778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:16.118670 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a0fc9-4ff2-4bc6-b0d3-c2ab6667e147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:16.119401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a0fc9-4ff2-4bc6-b0d3-c2ab6667e147 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:16.119689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a0fc9-4ff2-4bc6-b0d3-c2ab6667e147 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:16.119970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a0fc9-4ff2-4bc6-b0d3-c2ab6667e147 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:16.120734 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a0fc9-4ff2-4bc6-b0d3-c2ab6667e147 None admin] RBAC: Adding query filter params (name=member) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:16.121045 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a0fc9-4ff2-4bc6-b0d3-c2ab6667e147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:16.122428 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a0fc9-4ff2-4bc6-b0d3-c2ab6667e147 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:16.131351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 107/215] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:16.137164 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a0fc9-4ff2-4bc6-b0d3-c2ab6667e147 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:16.175440 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e00b5-6232-4415-91f8-4fca4a46f4e5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:16.177295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:16.218888 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045c6fc-3253-4293-991c-8f099805c74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:16.219980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045c6fc-3253-4293-991c-8f099805c74e None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:16.220550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045c6fc-3253-4293-991c-8f099805c74e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:16.221019 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045c6fc-3253-4293-991c-8f099805c74e None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:16.222033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045c6fc-3253-4293-991c-8f099805c74e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:16.223855 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045c6fc-3253-4293-991c-8f099805c74e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:16.249539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 109/216] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 14 20:20:16 2022] POST /identity/v3/users => generated 297 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:20:16.268664 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a0fc9-4ff2-4bc6-b0d3-c2ab6667e147 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:16.309580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34d820-4cce-4c9c-a143-9d7ffc24f485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:16.310483 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34d820-4cce-4c9c-a143-9d7ffc24f485 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:16.310768 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34d820-4cce-4c9c-a143-9d7ffc24f485 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:16.311131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34d820-4cce-4c9c-a143-9d7ffc24f485 None admin] PATH_INFO: `/v3/users/791cf2b89171496ba5e2bb26f9f2c217` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:16.327600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34d820-4cce-4c9c-a143-9d7ffc24f485 None admin] RBAC: Authorizing `identity:get_user(user_id=791cf2b89171496ba5e2bb26f9f2c217)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:16.329145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34d820-4cce-4c9c-a143-9d7ffc24f485 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:16.331057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 108/217] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:16 2022] GET /identity/v3/users/791cf2b89171496ba5e2bb26f9f2c217 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:16.336980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045c6fc-3253-4293-991c-8f099805c74e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:16.459198 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045c6fc-3253-4293-991c-8f099805c74e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:16.501503 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97145ed5-b319-4a00-b63b-6f484cafb5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:16.502532 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97145ed5-b319-4a00-b63b-6f484cafb5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:16.503041 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97145ed5-b319-4a00-b63b-6f484cafb5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:16.503529 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97145ed5-b319-4a00-b63b-6f484cafb5b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:16.534662 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:16.536987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafb4b7-af3b-437a-bcda-50a96703c088 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:16.537274 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafb4b7-af3b-437a-bcda-50a96703c088 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:16.537548 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafb4b7-af3b-437a-bcda-50a96703c088 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:16.539691 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 109/218] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:16.545090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563140b-a65f-4e7e-bcd4-900e8c9010fb None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:16.545385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563140b-a65f-4e7e-bcd4-900e8c9010fb None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:16.545902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563140b-a65f-4e7e-bcd4-900e8c9010fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:16.566768 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-6563140b-a65f-4e7e-bcd4-900e8c9010fb None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:16.567997 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6563140b-a65f-4e7e-bcd4-900e8c9010fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:16.590746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97145ed5-b319-4a00-b63b-6f484cafb5b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:16.592350 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97145ed5-b319-4a00-b63b-6f484cafb5b4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:16.603030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:16.672430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 110/219] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:20:16.697768 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563140b-a65f-4e7e-bcd4-900e8c9010fb None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:16.727452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 110/220] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:16.734489 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97145ed5-b319-4a00-b63b-6f484cafb5b4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:16.812163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:16.816114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563140b-a65f-4e7e-bcd4-900e8c9010fb None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:16.856004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c10b6-ab48-4977-8b8b-8dcc76eb6cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:16.857027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c10b6-ab48-4977-8b8b-8dcc76eb6cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:16.857145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c10b6-ab48-4977-8b8b-8dcc76eb6cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:16.857368 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c10b6-ab48-4977-8b8b-8dcc76eb6cd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:16.869196 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c10b6-ab48-4977-8b8b-8dcc76eb6cd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:16.871685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97145ed5-b319-4a00-b63b-6f484cafb5b4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:16.878491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-400c10b6-ab48-4977-8b8b-8dcc76eb6cd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:20:16.878491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:16.878491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:16.878491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:16.878491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:16.878491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:16.878491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:16.878491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:16.878491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:16.878491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:16.878491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:16.878491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:16.878491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:16.878491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:16.878491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:16.878491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:16.879422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:16.879422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:16.879422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:16.879422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:16.879422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:16.879422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:16.879422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:16.879422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:16.879422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:16.879422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:16.879422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:16.879422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:16.879422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:16.879422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:16.879422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:16.879422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:16.880207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:16.880207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:16.880207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:16.880207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:16.880207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:16.880207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:16.880207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:16.880207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:16.880207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:16.880207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:16.880207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:20:16.880207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:16.880207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 111/221] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:20:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:16.885867 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c10b6-ab48-4977-8b8b-8dcc76eb6cd5 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:16.910172 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:16.926181 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e1bcf-b0a5-46cb-b206-18c110c4c4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:16.927082 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e1bcf-b0a5-46cb-b206-18c110c4c4fb None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:16.927446 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e1bcf-b0a5-46cb-b206-18c110c4c4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:16.927740 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e1bcf-b0a5-46cb-b206-18c110c4c4fb None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:16.929898 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e1bcf-b0a5-46cb-b206-18c110c4c4fb None admin] RBAC: Authorizing `identity:get_project(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:16.931691 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e1bcf-b0a5-46cb-b206-18c110c4c4fb None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:16.933656 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 111/222] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:16 2022] GET /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:16.934984 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a44068-4353-4b14-a154-fbbf2e89ff2c None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:16.935303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a44068-4353-4b14-a154-fbbf2e89ff2c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:16.935575 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a44068-4353-4b14-a154-fbbf2e89ff2c None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:16.937603 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 112/223] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:16.942458 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a44068-4353-4b14-a154-fbbf2e89ff2c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:17.025069 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c10b6-ab48-4977-8b8b-8dcc76eb6cd5 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:17.069006 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a44068-4353-4b14-a154-fbbf2e89ff2c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:17.070058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56aca7-0f72-4201-baa8-46e49fca9f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:17.071040 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56aca7-0f72-4201-baa8-46e49fca9f17 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:17.072272 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56aca7-0f72-4201-baa8-46e49fca9f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:17.072664 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56aca7-0f72-4201-baa8-46e49fca9f17 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:17.073543 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56aca7-0f72-4201-baa8-46e49fca9f17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:17.073956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56aca7-0f72-4201-baa8-46e49fca9f17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:17.075486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56aca7-0f72-4201-baa8-46e49fca9f17 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:17.087138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 112/224] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:17.088925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266a245-9d40-4623-b6f3-bade122b29ac None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:17.089319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266a245-9d40-4623-b6f3-bade122b29ac None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:17.089699 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266a245-9d40-4623-b6f3-bade122b29ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:17.097687 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:17.109259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b752dc5-a486-4c55-98bb-d2cc71d0cdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:17.110364 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b752dc5-a486-4c55-98bb-d2cc71d0cdd5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:17.110825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b752dc5-a486-4c55-98bb-d2cc71d0cdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:17.111307 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b752dc5-a486-4c55-98bb-d2cc71d0cdd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:17.112032 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-7266a245-9d40-4623-b6f3-bade122b29ac None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:17.112577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b752dc5-a486-4c55-98bb-d2cc71d0cdd5 None admin] RBAC: Adding query filter params (role.id=b778e530d54948c7bc31f5b9231ea288, scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5, user.id=791cf2b89171496ba5e2bb26f9f2c217) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:17.113115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b752dc5-a486-4c55-98bb-d2cc71d0cdd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:17.113434 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-7266a245-9d40-4623-b6f3-bade122b29ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:17.115096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b752dc5-a486-4c55-98bb-d2cc71d0cdd5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:17.124151 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 113/225] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:16 2022] GET /identity/v3/role_assignments?user.id=791cf2b89171496ba5e2bb26f9f2c217&scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5&role.id=b778e530d54948c7bc31f5b9231ea288 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:17.124805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b752dc5-a486-4c55-98bb-d2cc71d0cdd5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:17.207227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 113/226] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:17.241788 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266a245-9d40-4623-b6f3-bade122b29ac None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:17.268816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b752dc5-a486-4c55-98bb-d2cc71d0cdd5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:17.273109 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:17.316957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399df49-9562-4ffb-ac70-3b1075bee06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:17.318116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399df49-9562-4ffb-ac70-3b1075bee06a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:17.320335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399df49-9562-4ffb-ac70-3b1075bee06a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:17.320775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399df49-9562-4ffb-ac70-3b1075bee06a None admin] PATH_INFO: `/v3/users/c420c0e76add44229d13aeb769ce9476` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:17.323623 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2399df49-9562-4ffb-ac70-3b1075bee06a None admin] RBAC: Authorizing `identity:get_user(user_id=c420c0e76add44229d13aeb769ce9476)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:17.329080 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2399df49-9562-4ffb-ac70-3b1075bee06a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:17.331180 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 114/227] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:17 2022] GET /identity/v3/users/c420c0e76add44229d13aeb769ce9476 => generated 283 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:17.340090 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399df49-9562-4ffb-ac70-3b1075bee06a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:17.417007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:17.454004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266a245-9d40-4623-b6f3-bade122b29ac None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:17.504425 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399df49-9562-4ffb-ac70-3b1075bee06a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:17.528189 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd464d-c3e7-4b97-96ef-95e23fdefd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:17.530485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd464d-c3e7-4b97-96ef-95e23fdefd3e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:17.531014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd464d-c3e7-4b97-96ef-95e23fdefd3e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:17.531602 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd464d-c3e7-4b97-96ef-95e23fdefd3e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:17.543596 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd464d-c3e7-4b97-96ef-95e23fdefd3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:17.555288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-5ffd464d-c3e7-4b97-96ef-95e23fdefd3e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:17.555288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:17.555288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:17.555288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:17.555288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:17.555288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:17.555288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:17.555288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:17.555288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:17.555288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:17.555288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:17.555288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:17.555288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:17.555288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:17.555288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:17.555288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:17.556212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:17.556212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:17.556212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:17.556212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:17.556212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:17.556212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:17.556212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:17.556212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:17.556212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:17.556212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:17.556212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:17.556212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:17.556212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:17.556212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:17.556212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:17.556212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:17.557004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:17.557004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:17.557004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:17.557004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:17.557004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:17.557004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:17.557004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:17.557004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:17.557004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:17.557004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:17.557004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:17.557004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:17.560248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 114/228] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:20:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:17.569840 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd464d-c3e7-4b97-96ef-95e23fdefd3e None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:17.602290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01f8dd-8340-429d-8ce8-c6e37275be48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:17.603799 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01f8dd-8340-429d-8ce8-c6e37275be48 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:17.604295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01f8dd-8340-429d-8ce8-c6e37275be48 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:17.604786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01f8dd-8340-429d-8ce8-c6e37275be48 None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:17.607671 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01f8dd-8340-429d-8ce8-c6e37275be48 None admin] RBAC: Authorizing `identity:get_project(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:17.610974 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01f8dd-8340-429d-8ce8-c6e37275be48 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:17.622509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 115/229] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:17 2022] GET /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:17.628741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01f8dd-8340-429d-8ce8-c6e37275be48 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:17.755333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:17.821520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd464d-c3e7-4b97-96ef-95e23fdefd3e None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:17.833827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01f8dd-8340-429d-8ce8-c6e37275be48 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:17.888867 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:17.904355 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ce454-0fb3-407e-be55-65476c5767a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:17.905672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ce454-0fb3-407e-be55-65476c5767a1 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:17.905955 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ce454-0fb3-407e-be55-65476c5767a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:17.906226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ce454-0fb3-407e-be55-65476c5767a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:17.907042 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ce454-0fb3-407e-be55-65476c5767a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:17.907353 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ce454-0fb3-407e-be55-65476c5767a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:17.910399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786571c-7522-4a4b-8333-864fad5b589b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:17.911420 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786571c-7522-4a4b-8333-864fad5b589b None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:17.911696 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786571c-7522-4a4b-8333-864fad5b589b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:17.911969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786571c-7522-4a4b-8333-864fad5b589b None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5/users/c420c0e76add44229d13aeb769ce9476/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:17.913238 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ce454-0fb3-407e-be55-65476c5767a1 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:17.917161 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786571c-7522-4a4b-8333-864fad5b589b None admin] RBAC: Authorizing `identity:create_grant(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5, user_id=c420c0e76add44229d13aeb769ce9476, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:17.918676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786571c-7522-4a4b-8333-864fad5b589b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:17.930012 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 115/230] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:20:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:17.944116 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ce454-0fb3-407e-be55-65476c5767a1 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:17.949623 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 116/231] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:20:17 2022] PUT /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5/users/c420c0e76add44229d13aeb769ce9476/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:20:18.119106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ce454-0fb3-407e-be55-65476c5767a1 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:18.172400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586eaa57-8a6d-4509-9173-ae9bb9e91b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:18.173368 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586eaa57-8a6d-4509-9173-ae9bb9e91b20 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:18.173697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586eaa57-8a6d-4509-9173-ae9bb9e91b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:18.173980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586eaa57-8a6d-4509-9173-ae9bb9e91b20 None admin] PATH_INFO: `/v3/users/c420c0e76add44229d13aeb769ce9476` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:18.175498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586eaa57-8a6d-4509-9173-ae9bb9e91b20 None admin] RBAC: Authorizing `identity:get_user(user_id=c420c0e76add44229d13aeb769ce9476)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:18.176969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586eaa57-8a6d-4509-9173-ae9bb9e91b20 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:18.178964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 116/232] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:17 2022] GET /identity/v3/users/c420c0e76add44229d13aeb769ce9476 => generated 283 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:18.184469 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786571c-7522-4a4b-8333-864fad5b589b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:18.340489 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786571c-7522-4a4b-8333-864fad5b589b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:18.379461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32caa35a-41f1-4905-834d-c09b6c2b93bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:18.380255 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32caa35a-41f1-4905-834d-c09b6c2b93bc None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:18.380539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32caa35a-41f1-4905-834d-c09b6c2b93bc None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:18.380817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32caa35a-41f1-4905-834d-c09b6c2b93bc None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:18.382046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32caa35a-41f1-4905-834d-c09b6c2b93bc None admin] RBAC: Authorizing `identity:get_project(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:18.383544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32caa35a-41f1-4905-834d-c09b6c2b93bc None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:18.390031 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 117/233] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:18 2022] GET /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:18.391494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586eaa57-8a6d-4509-9173-ae9bb9e91b20 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:18.395892 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:18.501781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586eaa57-8a6d-4509-9173-ae9bb9e91b20 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:18.541631 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1d316-8944-411b-8143-ef1f9def0e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:18.542551 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1d316-8944-411b-8143-ef1f9def0e94 None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:18.542833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1d316-8944-411b-8143-ef1f9def0e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:18.543139 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1d316-8944-411b-8143-ef1f9def0e94 None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5/users/c420c0e76add44229d13aeb769ce9476/roles/6b2eb3e4cefd4a2db4f18e04a158600a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:18.545484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1d316-8944-411b-8143-ef1f9def0e94 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5, user_id=c420c0e76add44229d13aeb769ce9476, role_id=6b2eb3e4cefd4a2db4f18e04a158600a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:18.547084 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1d316-8944-411b-8143-ef1f9def0e94 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:18.563983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 117/234] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:20:18 2022] PUT /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5/users/c420c0e76add44229d13aeb769ce9476/roles/6b2eb3e4cefd4a2db4f18e04a158600a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:20:18.995619 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3a2b2-e714-428f-8997-26bdb56631d1 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:18.996397 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3a2b2-e714-428f-8997-26bdb56631d1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:18.996397 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3a2b2-e714-428f-8997-26bdb56631d1 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:18.997989 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 118/235] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:19.003404 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cc7cb-367e-4ab1-8b1a-0793aede10f0 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:19.004154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cc7cb-367e-4ab1-8b1a-0793aede10f0 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:19.004154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cc7cb-367e-4ab1-8b1a-0793aede10f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:19.022784 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-ce3cc7cb-367e-4ab1-8b1a-0793aede10f0 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:19.028454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ce3cc7cb-367e-4ab1-8b1a-0793aede10f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:19.111339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:19.129807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 118/236] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:20:19.203142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34b577-d2d3-43dd-b161-f61fbf54d564 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:19.203463 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34b577-d2d3-43dd-b161-f61fbf54d564 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:19.203712 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34b577-d2d3-43dd-b161-f61fbf54d564 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:19.205531 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 119/237] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:19.206587 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cc7cb-367e-4ab1-8b1a-0793aede10f0 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:19.212505 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a84d9a-fc97-4747-b98f-abd007f58113 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:19.212802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a84d9a-fc97-4747-b98f-abd007f58113 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:19.213088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a84d9a-fc97-4747-b98f-abd007f58113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:19.243417 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-42a84d9a-fc97-4747-b98f-abd007f58113 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:19.244663 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-42a84d9a-fc97-4747-b98f-abd007f58113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:19.396522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cc7cb-367e-4ab1-8b1a-0793aede10f0 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:19.412940 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:19.453547 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 120/238] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:19.461700 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05226310-8708-4d22-91e7-220ec78e4acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:19.462726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05226310-8708-4d22-91e7-220ec78e4acc None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:19.463147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05226310-8708-4d22-91e7-220ec78e4acc None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:19.463529 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05226310-8708-4d22-91e7-220ec78e4acc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:19.479789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a84d9a-fc97-4747-b98f-abd007f58113 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:19.493238 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05226310-8708-4d22-91e7-220ec78e4acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:19.507919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-05226310-8708-4d22-91e7-220ec78e4acc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 20:20:19.507919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:19.507919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:19.507919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:19.507919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:19.507919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:19.507919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:19.507919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:19.507919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:19.507919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:19.507919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:19.507919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:19.507919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:19.507919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:19.507919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:19.507919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:19.508889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:19.508889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:19.508889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:19.508889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:19.508889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:19.508889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:19.508889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:19.508889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:19.508889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:19.508889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:19.508889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:19.508889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:19.508889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:19.508889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:19.508889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:19.508889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:19.509689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:19.509689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:19.509689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:19.509689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:19.509689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:19.509689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:19.509689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:19.509689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:19.509689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:19.509689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:19.509689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 20:20:19.509689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:19.509689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 119/239] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:19.517908 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05226310-8708-4d22-91e7-220ec78e4acc None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:19.606054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a84d9a-fc97-4747-b98f-abd007f58113 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:19.656359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927ea4e-2497-4784-9453-a603f83733ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:19.657831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927ea4e-2497-4784-9453-a603f83733ad None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:19.658333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927ea4e-2497-4784-9453-a603f83733ad None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:19.658804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927ea4e-2497-4784-9453-a603f83733ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:19.661116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927ea4e-2497-4784-9453-a603f83733ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:19.663310 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927ea4e-2497-4784-9453-a603f83733ad None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:19.665599 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 121/240] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:19.672690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff3fa4-e8bf-4873-8a66-4a5b87c5f670 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:19.673334 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff3fa4-e8bf-4873-8a66-4a5b87c5f670 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:19.673804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff3fa4-e8bf-4873-8a66-4a5b87c5f670 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:19.675478 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 122/241] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:19.675822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:19.677217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff3fa4-e8bf-4873-8a66-4a5b87c5f670 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:19.731027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05226310-8708-4d22-91e7-220ec78e4acc None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:19.758222 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:19.774234 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef15f96-ffe1-42c1-a2f3-27efcfdcd8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:19.775417 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef15f96-ffe1-42c1-a2f3-27efcfdcd8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:19.775918 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef15f96-ffe1-42c1-a2f3-27efcfdcd8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:19.776384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef15f96-ffe1-42c1-a2f3-27efcfdcd8c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:19.778373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef15f96-ffe1-42c1-a2f3-27efcfdcd8c0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:19.778666 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef15f96-ffe1-42c1-a2f3-27efcfdcd8c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:19.782202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef15f96-ffe1-42c1-a2f3-27efcfdcd8c0 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:19.804633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 120/242] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:19.804633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a813f-1339-448a-aa59-ca3a765e7f98 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:19.804633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a813f-1339-448a-aa59-ca3a765e7f98 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:19.804633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a813f-1339-448a-aa59-ca3a765e7f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:19.824599 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-8c3a813f-1339-448a-aa59-ca3a765e7f98 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:19.825801 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-8c3a813f-1339-448a-aa59-ca3a765e7f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:19.861476 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff3fa4-e8bf-4873-8a66-4a5b87c5f670 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:19.901591 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 121/243] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:19.907313 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a813f-1339-448a-aa59-ca3a765e7f98 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:19.951073 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746adaf-87a3-4962-a550-c98d632c4cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:19.953498 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746adaf-87a3-4962-a550-c98d632c4cee None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:19.954413 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746adaf-87a3-4962-a550-c98d632c4cee None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:19.955296 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746adaf-87a3-4962-a550-c98d632c4cee None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:19.957116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746adaf-87a3-4962-a550-c98d632c4cee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:19.957475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:19.968894 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746adaf-87a3-4962-a550-c98d632c4cee None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:20.023036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a813f-1339-448a-aa59-ca3a765e7f98 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:20.024386 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 123/244] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 14 20:20:19 2022] POST /identity/v3/users => generated 293 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:20:20.026025 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746adaf-87a3-4962-a550-c98d632c4cee None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:20.063193 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a2589-f868-4cb4-aa90-102ac4fd6171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:20.064560 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a2589-f868-4cb4-aa90-102ac4fd6171 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:20.065029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a2589-f868-4cb4-aa90-102ac4fd6171 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:20.065473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a2589-f868-4cb4-aa90-102ac4fd6171 None admin] PATH_INFO: `/v3/users/8ecee80fab8a4e8db799343ce838839a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:20.083268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a2589-f868-4cb4-aa90-102ac4fd6171 None admin] RBAC: Authorizing `identity:get_user(user_id=8ecee80fab8a4e8db799343ce838839a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:20.085276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a2589-f868-4cb4-aa90-102ac4fd6171 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:20.087399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 122/245] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:19 2022] GET /identity/v3/users/8ecee80fab8a4e8db799343ce838839a => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:20.093995 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a2589-f868-4cb4-aa90-102ac4fd6171 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:20.231452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746adaf-87a3-4962-a550-c98d632c4cee None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:20.253525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:20.274880 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a2589-f868-4cb4-aa90-102ac4fd6171 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:20.276629 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:20.319752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7c5cb-0861-447f-9fac-d4d9f2386e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:20.320572 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7c5cb-0861-447f-9fac-d4d9f2386e11 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:20.321022 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7c5cb-0861-447f-9fac-d4d9f2386e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:20.321444 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7c5cb-0861-447f-9fac-d4d9f2386e11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:20.339958 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e7c5cb-0861-447f-9fac-d4d9f2386e11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:20.352003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-95e7c5cb-0861-447f-9fac-d4d9f2386e11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:20.352003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:20.352003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:20.352003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:20.352003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:20.352003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:20.352003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:20.352003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:20.352003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:20.352003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:20.352003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:20.352003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:20.352003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:20.352003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:20.352003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:20.352003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:20.353112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:20.353112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:20.353112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:20.353112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:20.353112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:20.353112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:20.353112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:20.353112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:20.353112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:20.353112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:20.353112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:20.353112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:20.353112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:20.353112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:20.353112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:20.353112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:20.354152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:20.354152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:20.354152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:20.354152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:20.354152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:20.354152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:20.354152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:20.354152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:20.354152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:20.354152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:20.354152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:20.354152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:20.356558 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 124/246] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:20.356682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd122e-2608-47ac-bdc2-9373de82a333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:20.357503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd122e-2608-47ac-bdc2-9373de82a333 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:20.358450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd122e-2608-47ac-bdc2-9373de82a333 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:20.358805 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd122e-2608-47ac-bdc2-9373de82a333 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:20.364991 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7c5cb-0861-447f-9fac-d4d9f2386e11 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:20.466643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd122e-2608-47ac-bdc2-9373de82a333 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:20.469175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd122e-2608-47ac-bdc2-9373de82a333 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:20.503771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7c5cb-0861-447f-9fac-d4d9f2386e11 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:20.549258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc3d88-0830-48e1-8aba-0ef5be41e420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:20.550609 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc3d88-0830-48e1-8aba-0ef5be41e420 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:20.550852 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc3d88-0830-48e1-8aba-0ef5be41e420 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:20.551337 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc3d88-0830-48e1-8aba-0ef5be41e420 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:20.552223 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc3d88-0830-48e1-8aba-0ef5be41e420 None admin] RBAC: Adding query filter params (name=member) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:20.552559 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc3d88-0830-48e1-8aba-0ef5be41e420 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:20.554344 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc3d88-0830-48e1-8aba-0ef5be41e420 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:20.564355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 125/247] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:20.571289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc3d88-0830-48e1-8aba-0ef5be41e420 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:20.592862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 123/248] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:20.595077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd122e-2608-47ac-bdc2-9373de82a333 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:20.597504 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:20.624571 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:20.727305 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc3d88-0830-48e1-8aba-0ef5be41e420 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:20.801294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534d68d-ecf3-4db4-87b9-d1926bbcd3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:20.802129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534d68d-ecf3-4db4-87b9-d1926bbcd3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:20.802415 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534d68d-ecf3-4db4-87b9-d1926bbcd3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:20.802695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534d68d-ecf3-4db4-87b9-d1926bbcd3d4 None admin] PATH_INFO: `/v3/users/791cf2b89171496ba5e2bb26f9f2c217` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:20.805316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534d68d-ecf3-4db4-87b9-d1926bbcd3d4 None admin] RBAC: Authorizing `identity:get_user(user_id=791cf2b89171496ba5e2bb26f9f2c217)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:20.807166 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534d68d-ecf3-4db4-87b9-d1926bbcd3d4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:20.811734 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 126/249] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:20 2022] GET /identity/v3/users/791cf2b89171496ba5e2bb26f9f2c217 => generated 297 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:20.812956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca90b6-23a2-4477-b816-e66b08f926ce None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:20.813164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca90b6-23a2-4477-b816-e66b08f926ce None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:20.813446 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca90b6-23a2-4477-b816-e66b08f926ce None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:20.814802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 127/250] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:20.821335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca90b6-23a2-4477-b816-e66b08f926ce None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:20.832147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd122e-2608-47ac-bdc2-9373de82a333 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:20.905794 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520bbca-6267-4e45-8e7e-1d6b67901924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:20.907461 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520bbca-6267-4e45-8e7e-1d6b67901924 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:20.908054 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520bbca-6267-4e45-8e7e-1d6b67901924 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:20.908527 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520bbca-6267-4e45-8e7e-1d6b67901924 None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:20.916785 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520bbca-6267-4e45-8e7e-1d6b67901924 None admin] RBAC: Authorizing `identity:get_project(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:20.918612 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520bbca-6267-4e45-8e7e-1d6b67901924 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:20.925898 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 124/251] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:20 2022] GET /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5 => generated 302 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:20.927864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574302c-7a06-4abf-b6fb-d07c91a64a65 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:20.928376 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574302c-7a06-4abf-b6fb-d07c91a64a65 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:20.928847 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574302c-7a06-4abf-b6fb-d07c91a64a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:20.958957 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-c574302c-7a06-4abf-b6fb-d07c91a64a65 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:20.960599 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-c574302c-7a06-4abf-b6fb-d07c91a64a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:20.991463 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:20.997738 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca90b6-23a2-4477-b816-e66b08f926ce None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:21.044968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0324c9-faf0-4d0a-a4d9-5c570d618597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:21.045898 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0324c9-faf0-4d0a-a4d9-5c570d618597 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:21.046178 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0324c9-faf0-4d0a-a4d9-5c570d618597 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:21.046448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0324c9-faf0-4d0a-a4d9-5c570d618597 None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:21.047874 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0324c9-faf0-4d0a-a4d9-5c570d618597 None admin] RBAC: Authorizing `identity:get_project(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:21.049508 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0324c9-faf0-4d0a-a4d9-5c570d618597 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:21.051536 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 128/252] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:20 2022] GET /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:21.052380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0324c9-faf0-4d0a-a4d9-5c570d618597 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:21.066959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:21.073118 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 125/253] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:21.083291 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2043fba-51d1-4e6a-89c2-3c9c818ead6b None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:21.083687 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2043fba-51d1-4e6a-89c2-3c9c818ead6b None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:21.083973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2043fba-51d1-4e6a-89c2-3c9c818ead6b None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:21.085870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 126/254] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:21 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:21.086661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2043fba-51d1-4e6a-89c2-3c9c818ead6b None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:21.176384 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0324c9-faf0-4d0a-a4d9-5c570d618597 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:21.232252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6992ec3-3c2f-47a7-8767-4ceeb2e12fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:21.233324 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6992ec3-3c2f-47a7-8767-4ceeb2e12fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:21.233609 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6992ec3-3c2f-47a7-8767-4ceeb2e12fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:21.233883 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6992ec3-3c2f-47a7-8767-4ceeb2e12fb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:21.234799 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6992ec3-3c2f-47a7-8767-4ceeb2e12fb9 None admin] RBAC: Adding query filter params (role.id=a7d155dd59914d22a32012302fa6d2d2, scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5, user.id=8ecee80fab8a4e8db799343ce838839a) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:21.235348 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6992ec3-3c2f-47a7-8767-4ceeb2e12fb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:21.237333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6992ec3-3c2f-47a7-8767-4ceeb2e12fb9 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:21.245162 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 129/255] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:21 2022] GET /identity/v3/role_assignments?user.id=8ecee80fab8a4e8db799343ce838839a&scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5&role.id=a7d155dd59914d22a32012302fa6d2d2 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:21.246504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1391960-f8b2-4f66-8c07-38e4b6d280c6 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:21.246794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1391960-f8b2-4f66-8c07-38e4b6d280c6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:21.247156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1391960-f8b2-4f66-8c07-38e4b6d280c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:21.254157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2043fba-51d1-4e6a-89c2-3c9c818ead6b None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:21.269765 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d1391960-f8b2-4f66-8c07-38e4b6d280c6 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:21.271075 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d1391960-f8b2-4f66-8c07-38e4b6d280c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:21.308719 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:21.321543 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36a555-26f0-491e-a92b-fdc5122c10db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:21.323179 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36a555-26f0-491e-a92b-fdc5122c10db None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:21.323583 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36a555-26f0-491e-a92b-fdc5122c10db None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:21.323921 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36a555-26f0-491e-a92b-fdc5122c10db None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5/users/791cf2b89171496ba5e2bb26f9f2c217/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:21.328534 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36a555-26f0-491e-a92b-fdc5122c10db None admin] RBAC: Authorizing `identity:create_grant(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5, user_id=791cf2b89171496ba5e2bb26f9f2c217, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:21.331303 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36a555-26f0-491e-a92b-fdc5122c10db None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:21.361895 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 127/256] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:20:21 2022] PUT /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5/users/791cf2b89171496ba5e2bb26f9f2c217/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:20:21.362856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36a555-26f0-491e-a92b-fdc5122c10db None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:21.370507 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 130/257] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:21.397318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1391960-f8b2-4f66-8c07-38e4b6d280c6 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:21.469599 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:21.521218 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36a555-26f0-491e-a92b-fdc5122c10db None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:21.531836 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1391960-f8b2-4f66-8c07-38e4b6d280c6 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:21.570398 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782da294-3883-468e-94e1-016a4a9f0c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:21.571207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782da294-3883-468e-94e1-016a4a9f0c63 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:21.571495 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782da294-3883-468e-94e1-016a4a9f0c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:21.571767 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782da294-3883-468e-94e1-016a4a9f0c63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:21.575076 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56764167-6cf8-4858-a1fe-984cbba7c21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:21.576275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56764167-6cf8-4858-a1fe-984cbba7c21f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:21.576798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56764167-6cf8-4858-a1fe-984cbba7c21f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:21.577272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56764167-6cf8-4858-a1fe-984cbba7c21f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:21.584770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782da294-3883-468e-94e1-016a4a9f0c63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:21.592920 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56764167-6cf8-4858-a1fe-984cbba7c21f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:21.600104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-782da294-3883-468e-94e1-016a4a9f0c63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:20:21.600104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:21.600104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:21.600104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:21.600104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:21.600104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:21.600104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:21.600104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:21.600104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:21.600104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:21.600104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:21.600104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:21.600104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:21.600104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:21.600104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:21.600104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:21.601641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:21.601641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:21.601641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:21.601641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:21.601641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:21.601641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:21.601641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:21.601641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:21.601641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:21.601641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:21.601641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:21.601641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:21.601641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:21.601641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:21.601641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:21.601641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:21.602436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:21.602436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:21.602436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:21.602436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:21.602436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:21.602436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:21.602436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:21.602436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:21.602436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:21.602436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:21.602436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:20:21.602436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:21.604749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 128/258] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:20:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:21.607320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-56764167-6cf8-4858-a1fe-984cbba7c21f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:21.607320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:21.607320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:21.607320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:21.607320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:21.607320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:21.607320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:21.607320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:21.607320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:21.607320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:21.607320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:21.607320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:21.607320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:21.607320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:21.607320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:21.607320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:21.608885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:21.608885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:21.608885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:21.608885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:21.608885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:21.608885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:21.608885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:21.608885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:21.608885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:21.608885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:21.608885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:21.608885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:21.608885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:21.608885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:21.608885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:21.608885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:21.609804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:21.609804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:21.609804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:21.609804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:21.609804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:21.609804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:21.609804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:21.609804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:21.609804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:21.609804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:21.609804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:21.609804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:21.612598 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782da294-3883-468e-94e1-016a4a9f0c63 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:21.613005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 131/259] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:20:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:21.619807 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56764167-6cf8-4858-a1fe-984cbba7c21f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:21.669106 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:21.815018 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782da294-3883-468e-94e1-016a4a9f0c63 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:21.838348 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:21.854905 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56764167-6cf8-4858-a1fe-984cbba7c21f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:21.855244 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81fe90-ce4b-43bb-9856-6ec2c77410fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:21.855892 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81fe90-ce4b-43bb-9856-6ec2c77410fc None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:21.856179 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81fe90-ce4b-43bb-9856-6ec2c77410fc None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:21.856459 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81fe90-ce4b-43bb-9856-6ec2c77410fc None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:21.857346 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81fe90-ce4b-43bb-9856-6ec2c77410fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:21.857621 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81fe90-ce4b-43bb-9856-6ec2c77410fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:21.859183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81fe90-ce4b-43bb-9856-6ec2c77410fc None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:21.868966 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 129/260] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:21.880960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81fe90-ce4b-43bb-9856-6ec2c77410fc None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:21.972451 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8139543-846d-4c1e-97f3-0b043c68e6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:21.973238 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8139543-846d-4c1e-97f3-0b043c68e6fd None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:21.973528 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8139543-846d-4c1e-97f3-0b043c68e6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:21.973802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8139543-846d-4c1e-97f3-0b043c68e6fd None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:21.974615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8139543-846d-4c1e-97f3-0b043c68e6fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:21.974923 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8139543-846d-4c1e-97f3-0b043c68e6fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:21.976504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8139543-846d-4c1e-97f3-0b043c68e6fd None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:22.005443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 132/261] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:20:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:22.032168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8139543-846d-4c1e-97f3-0b043c68e6fd None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:22.050735 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81fe90-ce4b-43bb-9856-6ec2c77410fc None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:22.116046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29a91f-2e8a-4663-93ef-ec6bb983a53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:22.116870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29a91f-2e8a-4663-93ef-ec6bb983a53e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:22.117157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29a91f-2e8a-4663-93ef-ec6bb983a53e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:22.117436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29a91f-2e8a-4663-93ef-ec6bb983a53e None admin] PATH_INFO: `/v3/users/c420c0e76add44229d13aeb769ce9476` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:22.118676 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29a91f-2e8a-4663-93ef-ec6bb983a53e None admin] RBAC: Authorizing `identity:get_user(user_id=c420c0e76add44229d13aeb769ce9476)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:22.120192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29a91f-2e8a-4663-93ef-ec6bb983a53e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:22.122139 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 130/262] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:21 2022] GET /identity/v3/users/c420c0e76add44229d13aeb769ce9476 => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:22.129970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29a91f-2e8a-4663-93ef-ec6bb983a53e None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:22.215336 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:22.323687 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29a91f-2e8a-4663-93ef-ec6bb983a53e None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:22.335338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:22.370975 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4c402-7333-4f31-92ee-dc60effd4150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:22.371995 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4c402-7333-4f31-92ee-dc60effd4150 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:22.372478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4c402-7333-4f31-92ee-dc60effd4150 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:22.372968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4c402-7333-4f31-92ee-dc60effd4150 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:22.377126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8139543-846d-4c1e-97f3-0b043c68e6fd None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:22.444773 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30115608-7899-43a9-b6b5-b76c3b655853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:22.446055 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30115608-7899-43a9-b6b5-b76c3b655853 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:22.448504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30115608-7899-43a9-b6b5-b76c3b655853 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:22.449037 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30115608-7899-43a9-b6b5-b76c3b655853 None admin] PATH_INFO: `/v3/users/c420c0e76add44229d13aeb769ce9476` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:22.450553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30115608-7899-43a9-b6b5-b76c3b655853 None admin] RBAC: Authorizing `identity:get_user(user_id=c420c0e76add44229d13aeb769ce9476)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:22.452509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30115608-7899-43a9-b6b5-b76c3b655853 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:22.455695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 133/263] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:22 2022] GET /identity/v3/users/c420c0e76add44229d13aeb769ce9476 => generated 283 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:22.463849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30115608-7899-43a9-b6b5-b76c3b655853 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:22.503641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c4c402-7333-4f31-92ee-dc60effd4150 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:22.505147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c4c402-7333-4f31-92ee-dc60effd4150 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:22.600009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:22.631670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30115608-7899-43a9-b6b5-b76c3b655853 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:22.667624 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89dd4d7-b0fb-40f4-a8cf-5357a70201ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:22.668489 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89dd4d7-b0fb-40f4-a8cf-5357a70201ce None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:22.668865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89dd4d7-b0fb-40f4-a8cf-5357a70201ce None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:22.669268 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89dd4d7-b0fb-40f4-a8cf-5357a70201ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:22.693508 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:22.732089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 131/264] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:22.738456 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4c402-7333-4f31-92ee-dc60effd4150 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:22.744842 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89dd4d7-b0fb-40f4-a8cf-5357a70201ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:22.746556 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89dd4d7-b0fb-40f4-a8cf-5357a70201ce None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:22.850633 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 134/265] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:22.857481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89dd4d7-b0fb-40f4-a8cf-5357a70201ce None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:22.951046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4c402-7333-4f31-92ee-dc60effd4150 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:23.003501 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e667ef-4bec-4922-a055-ccf5a0a85e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:23.004315 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:23.004435 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e667ef-4bec-4922-a055-ccf5a0a85e4d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:23.004654 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e667ef-4bec-4922-a055-ccf5a0a85e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:23.004953 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e667ef-4bec-4922-a055-ccf5a0a85e4d None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:23.006168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e667ef-4bec-4922-a055-ccf5a0a85e4d None admin] RBAC: Authorizing `identity:get_project(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:23.007711 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e667ef-4bec-4922-a055-ccf5a0a85e4d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:23.009727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 132/266] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:22 2022] GET /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:23.016038 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e667ef-4bec-4922-a055-ccf5a0a85e4d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:23.036823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:23.121661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e667ef-4bec-4922-a055-ccf5a0a85e4d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:23.145614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89dd4d7-b0fb-40f4-a8cf-5357a70201ce None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:23.162962 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6d73c-4fb8-4535-888c-8960240ee69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:23.162962 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6d73c-4fb8-4535-888c-8960240ee69d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:23.162962 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6d73c-4fb8-4535-888c-8960240ee69d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:23.162962 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6d73c-4fb8-4535-888c-8960240ee69d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:23.163379 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6d73c-4fb8-4535-888c-8960240ee69d None admin] RBAC: Adding query filter params (role.id=5ca112885467422684e134cb0f94865a, scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5, user.id=c420c0e76add44229d13aeb769ce9476) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:23.163379 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6d73c-4fb8-4535-888c-8960240ee69d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:23.163379 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6d73c-4fb8-4535-888c-8960240ee69d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:23.168108 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 133/267] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:23 2022] GET /identity/v3/role_assignments?user.id=c420c0e76add44229d13aeb769ce9476&scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5&role.id=5ca112885467422684e134cb0f94865a => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:23.222035 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ebbf9b-29a9-443a-90f7-660075230927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:23.223492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebbf9b-29a9-443a-90f7-660075230927 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:23.224018 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebbf9b-29a9-443a-90f7-660075230927 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:23.224503 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebbf9b-29a9-443a-90f7-660075230927 None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:23.231995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ebbf9b-29a9-443a-90f7-660075230927 None admin] RBAC: Authorizing `identity:get_project(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:23.233893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ebbf9b-29a9-443a-90f7-660075230927 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:23.236371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 135/268] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:22 2022] GET /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5 => generated 302 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:23.242027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6d73c-4fb8-4535-888c-8960240ee69d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:23.358089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:23.379300 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6d73c-4fb8-4535-888c-8960240ee69d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:23.418774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52ac00-817c-4490-8d68-e8892501d925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:23.419511 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52ac00-817c-4490-8d68-e8892501d925 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:23.419777 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52ac00-817c-4490-8d68-e8892501d925 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:23.420061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52ac00-817c-4490-8d68-e8892501d925 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:23.421009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52ac00-817c-4490-8d68-e8892501d925 None admin] RBAC: Adding query filter params (role.id=6b2eb3e4cefd4a2db4f18e04a158600a, scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5, user.id=c420c0e76add44229d13aeb769ce9476) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:23.421322 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52ac00-817c-4490-8d68-e8892501d925 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:23.422799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52ac00-817c-4490-8d68-e8892501d925 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:23.429959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 134/269] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:23 2022] GET /identity/v3/role_assignments?user.id=c420c0e76add44229d13aeb769ce9476&scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5&role.id=6b2eb3e4cefd4a2db4f18e04a158600a => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:23.436647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e57a2-02cd-468a-bc5b-d18d8c4090ce None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:23.437614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e57a2-02cd-468a-bc5b-d18d8c4090ce None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:23.438143 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e57a2-02cd-468a-bc5b-d18d8c4090ce None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:23.441868 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 136/270] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:23.448193 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0460b-79c2-46b4-b07b-ed7ad1800be4 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:23.448490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0460b-79c2-46b4-b07b-ed7ad1800be4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:23.448748 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0460b-79c2-46b4-b07b-ed7ad1800be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:23.468972 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-89f0460b-79c2-46b4-b07b-ed7ad1800be4 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:23.470578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-89f0460b-79c2-46b4-b07b-ed7ad1800be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:23.552998 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca59b46-8a22-451f-a23a-0551aa55585a None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:23.553200 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca59b46-8a22-451f-a23a-0551aa55585a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:23.553412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca59b46-8a22-451f-a23a-0551aa55585a None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:23.556249 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 137/271] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:23.561516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed2918-d9ee-4595-8438-f8bdac1df05e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:23.561804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed2918-d9ee-4595-8438-f8bdac1df05e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:23.562076 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed2918-d9ee-4595-8438-f8bdac1df05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:23.567071 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 135/272] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:23.596449 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e9ed2918-d9ee-4595-8438-f8bdac1df05e None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:23.597684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e9ed2918-d9ee-4595-8438-f8bdac1df05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:23.626393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0460b-79c2-46b4-b07b-ed7ad1800be4 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:23.713727 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 138/273] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:20:23.739161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:23.742173 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed2918-d9ee-4595-8438-f8bdac1df05e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:23.781183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0460b-79c2-46b4-b07b-ed7ad1800be4 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:23.819721 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:23.830260 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54700ba7-55b3-487f-b758-d90fc156058a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:23.832040 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54700ba7-55b3-487f-b758-d90fc156058a None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:23.832538 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54700ba7-55b3-487f-b758-d90fc156058a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:23.833542 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54700ba7-55b3-487f-b758-d90fc156058a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:23.850788 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54700ba7-55b3-487f-b758-d90fc156058a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:23.860101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-54700ba7-55b3-487f-b758-d90fc156058a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:23.860101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:23.860101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:23.860101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:23.860101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:23.860101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:23.860101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:23.860101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:23.860101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:23.860101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:23.860101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:23.860101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:23.860101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:23.860101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:23.860101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:23.860101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:23.861851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:23.861851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:23.861851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:23.861851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:23.861851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:23.861851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:23.861851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:23.861851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:23.861851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:23.861851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:23.861851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:23.861851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:23.861851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:23.861851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:23.861851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:23.861851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:23.863496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:23.863496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:23.863496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:23.863496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:23.863496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:23.863496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:23.863496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:23.863496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:23.863496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:23.863496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:23.863496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:23.863496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:23.863496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 136/274] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:23.868175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54700ba7-55b3-487f-b758-d90fc156058a None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:23.886291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed2918-d9ee-4595-8438-f8bdac1df05e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:23.932203 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f4537-7d0d-4266-80dc-72dea4a1c226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:23.933005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f4537-7d0d-4266-80dc-72dea4a1c226 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:23.933289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f4537-7d0d-4266-80dc-72dea4a1c226 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:23.933561 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f4537-7d0d-4266-80dc-72dea4a1c226 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:23.934738 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f4537-7d0d-4266-80dc-72dea4a1c226 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:23.936274 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f4537-7d0d-4266-80dc-72dea4a1c226 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:23.938210 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 139/275] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:23.944248 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f4537-7d0d-4266-80dc-72dea4a1c226 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:24.008950 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54700ba7-55b3-487f-b758-d90fc156058a None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:24.054820 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f4537-7d0d-4266-80dc-72dea4a1c226 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:24.058601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a08b2-24e4-4427-8215-3b26be186fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:24.059689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a08b2-24e4-4427-8215-3b26be186fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:24.060210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a08b2-24e4-4427-8215-3b26be186fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:24.060695 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a08b2-24e4-4427-8215-3b26be186fe6 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:24.062359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a08b2-24e4-4427-8215-3b26be186fe6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:24.062953 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a08b2-24e4-4427-8215-3b26be186fe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:24.065392 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a08b2-24e4-4427-8215-3b26be186fe6 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:24.069425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:24.077352 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:24.082228 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 137/276] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:24.092022 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a08b2-24e4-4427-8215-3b26be186fe6 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:24.106558 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c01f4-12bd-49e0-8a5f-d46d60408ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:24.107360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c01f4-12bd-49e0-8a5f-d46d60408ff8 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:24.108063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c01f4-12bd-49e0-8a5f-d46d60408ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:24.108063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c01f4-12bd-49e0-8a5f-d46d60408ff8 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:24.108832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c01f4-12bd-49e0-8a5f-d46d60408ff8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:24.110451 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c01f4-12bd-49e0-8a5f-d46d60408ff8 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:24.141400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 140/277] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 14 20:20:23 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:20:24.225624 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a08b2-24e4-4427-8215-3b26be186fe6 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:24.231662 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff4435-2785-4be8-b972-faec081a0fd2 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:24.231990 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff4435-2785-4be8-b972-faec081a0fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:24.232257 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff4435-2785-4be8-b972-faec081a0fd2 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:24.233753 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 141/278] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:24.239667 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49272be7-8750-4bb2-9915-c0782f63e192 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:24.240293 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49272be7-8750-4bb2-9915-c0782f63e192 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:24.240647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49272be7-8750-4bb2-9915-c0782f63e192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:24.261627 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-49272be7-8750-4bb2-9915-c0782f63e192 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:24.263381 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-49272be7-8750-4bb2-9915-c0782f63e192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:24.271130 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca26df-63f0-4a0c-8143-e40b8308ed62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:24.272040 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca26df-63f0-4a0c-8143-e40b8308ed62 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:24.272329 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca26df-63f0-4a0c-8143-e40b8308ed62 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:24.272621 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca26df-63f0-4a0c-8143-e40b8308ed62 None admin] PATH_INFO: `/v3/users/c345adca638d46248bcc4415a55f6f76` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:24.291999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca26df-63f0-4a0c-8143-e40b8308ed62 None admin] RBAC: Authorizing `identity:get_user(user_id=c345adca638d46248bcc4415a55f6f76)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:24.293638 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca26df-63f0-4a0c-8143-e40b8308ed62 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:24.295665 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 138/279] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:24 2022] GET /identity/v3/users/c345adca638d46248bcc4415a55f6f76 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:24.301620 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca26df-63f0-4a0c-8143-e40b8308ed62 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:24.351428 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 142/280] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:20:24.371450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:24.376341 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49272be7-8750-4bb2-9915-c0782f63e192 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:24.437273 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca26df-63f0-4a0c-8143-e40b8308ed62 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:24.481249 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51ab3f-af21-4597-9331-60eb7425cac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:24.482047 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51ab3f-af21-4597-9331-60eb7425cac3 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:24.482338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51ab3f-af21-4597-9331-60eb7425cac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:24.482620 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51ab3f-af21-4597-9331-60eb7425cac3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:24.484370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51ab3f-af21-4597-9331-60eb7425cac3 None admin] RBAC: Adding query filter params (role.id=b778e530d54948c7bc31f5b9231ea288, user.id=c345adca638d46248bcc4415a55f6f76, scope.system=all) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:24.484748 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51ab3f-af21-4597-9331-60eb7425cac3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:24.486416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51ab3f-af21-4597-9331-60eb7425cac3 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:24.492317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 139/281] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Jul 14 20:20:24 2022] GET /identity/v3/role_assignments?user.id=c345adca638d46248bcc4415a55f6f76&scope.system=all&role.id=b778e530d54948c7bc31f5b9231ea288 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:24.506461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49272be7-8750-4bb2-9915-c0782f63e192 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:24.510776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:24.553827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6733dc-c478-4f32-b5c2-1accf46a0a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:24.555129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6733dc-c478-4f32-b5c2-1accf46a0a31 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:24.555637 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6733dc-c478-4f32-b5c2-1accf46a0a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:24.556132 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6733dc-c478-4f32-b5c2-1accf46a0a31 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:24.573600 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6733dc-c478-4f32-b5c2-1accf46a0a31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:24.582108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-eb6733dc-c478-4f32-b5c2-1accf46a0a31 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 20:20:24.582108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:24.582108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:24.582108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:24.582108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:24.582108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:24.582108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:24.582108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:24.582108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:24.582108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:24.582108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:24.582108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:24.582108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:24.582108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:24.582108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:24.582108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:24.583127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:24.583127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:24.583127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:24.583127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:24.583127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:24.583127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:24.583127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:24.583127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:24.583127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:24.583127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:24.583127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:24.583127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:24.583127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:24.583127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:24.583127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:24.583127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:24.584033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:24.584033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:24.584033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:24.584033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:24.584033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:24.584033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:24.584033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:24.584033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:24.584033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:24.584033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:24.584033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 20:20:24.584033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:24.585596 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 143/282] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:24.592390 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51ab3f-af21-4597-9331-60eb7425cac3 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:24.683947 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7c5ec-c6c5-4eba-a0fe-61dd41ac1de5 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:24.684264 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7c5ec-c6c5-4eba-a0fe-61dd41ac1de5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:24.684513 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7c5ec-c6c5-4eba-a0fe-61dd41ac1de5 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:24.687716 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 144/283] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:20:24.693350 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda9bbc-9a40-4c58-9597-3902716137b8 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:24.693717 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda9bbc-9a40-4c58-9597-3902716137b8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:24.693949 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda9bbc-9a40-4c58-9597-3902716137b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:24.723509 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51ab3f-af21-4597-9331-60eb7425cac3 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:24.729440 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-edda9bbc-9a40-4c58-9597-3902716137b8 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:24.730649 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-edda9bbc-9a40-4c58-9597-3902716137b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:24.750701 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:24.781260 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0a774-bc0d-4f16-b379-dd11dd30d4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:24.782330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0a774-bc0d-4f16-b379-dd11dd30d4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:24.782809 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0a774-bc0d-4f16-b379-dd11dd30d4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:24.783300 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0a774-bc0d-4f16-b379-dd11dd30d4f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:24.785144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0a774-bc0d-4f16-b379-dd11dd30d4f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:24.789993 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0a774-bc0d-4f16-b379-dd11dd30d4f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:24.791714 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0a774-bc0d-4f16-b379-dd11dd30d4f7 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:24.810542 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 140/284] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:24.816718 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0a774-bc0d-4f16-b379-dd11dd30d4f7 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:24.825876 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 145/285] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:20:24.849607 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda9bbc-9a40-4c58-9597-3902716137b8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:24.951413 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:24.960940 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda9bbc-9a40-4c58-9597-3902716137b8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:25.002001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739567e5-a91d-4d1f-abe6-cfca21dec44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:25.002834 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739567e5-a91d-4d1f-abe6-cfca21dec44f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:25.003154 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739567e5-a91d-4d1f-abe6-cfca21dec44f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:25.003436 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739567e5-a91d-4d1f-abe6-cfca21dec44f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:25.023966 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739567e5-a91d-4d1f-abe6-cfca21dec44f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:25.045903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-739567e5-a91d-4d1f-abe6-cfca21dec44f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:25.045903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:25.045903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:25.045903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:25.045903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:25.045903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:25.045903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:25.045903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:25.045903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:25.045903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:25.045903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:25.045903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:25.045903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:25.045903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:25.045903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:25.045903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:25.046999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:25.046999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:25.046999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:25.046999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:25.046999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:25.046999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:25.046999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:25.046999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:25.046999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:25.046999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:25.046999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:25.046999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:25.046999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:25.046999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:25.046999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:25.046999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:25.047880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:25.047880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:25.047880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:25.047880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:25.047880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:25.047880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:25.047880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:25.047880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:25.047880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:25.047880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:25.047880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:25.047880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:25.051145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 146/286] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:25.062225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0a774-bc0d-4f16-b379-dd11dd30d4f7 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:25.065933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739567e5-a91d-4d1f-abe6-cfca21dec44f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:25.106522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e4436-07a5-4f30-a681-8fd682459377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:25.107461 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e4436-07a5-4f30-a681-8fd682459377 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:25.108630 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e4436-07a5-4f30-a681-8fd682459377 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:25.108929 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e4436-07a5-4f30-a681-8fd682459377 None admin] PATH_INFO: `/v3/users/8ecee80fab8a4e8db799343ce838839a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:25.110205 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e4436-07a5-4f30-a681-8fd682459377 None admin] RBAC: Authorizing `identity:get_user(user_id=8ecee80fab8a4e8db799343ce838839a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:25.111720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e4436-07a5-4f30-a681-8fd682459377 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:25.113611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 141/287] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:24 2022] GET /identity/v3/users/8ecee80fab8a4e8db799343ce838839a => generated 297 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:25.118767 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e4436-07a5-4f30-a681-8fd682459377 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:25.227430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:25.270767 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739567e5-a91d-4d1f-abe6-cfca21dec44f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:25.307763 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40696a-729b-4d43-9625-47b551c64e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:25.308436 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40696a-729b-4d43-9625-47b551c64e42 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:25.308879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40696a-729b-4d43-9625-47b551c64e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:25.309159 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40696a-729b-4d43-9625-47b551c64e42 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:25.310077 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40696a-729b-4d43-9625-47b551c64e42 None admin] RBAC: Adding query filter params (name=member) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:25.310383 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40696a-729b-4d43-9625-47b551c64e42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:25.312008 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40696a-729b-4d43-9625-47b551c64e42 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:25.321611 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 147/288] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:25.327080 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e4436-07a5-4f30-a681-8fd682459377 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:25.328121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40696a-729b-4d43-9625-47b551c64e42 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:25.347319 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:25.394854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e90e19-d8b7-4491-acc8-0d8df72f5b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:25.395787 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e90e19-d8b7-4491-acc8-0d8df72f5b57 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:25.396071 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e90e19-d8b7-4491-acc8-0d8df72f5b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:25.396349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e90e19-d8b7-4491-acc8-0d8df72f5b57 None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:25.400372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e90e19-d8b7-4491-acc8-0d8df72f5b57 None admin] RBAC: Authorizing `identity:get_project(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:25.401871 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e90e19-d8b7-4491-acc8-0d8df72f5b57 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:25.403983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 142/289] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:25 2022] GET /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5 => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:25.410403 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e90e19-d8b7-4491-acc8-0d8df72f5b57 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:25.479175 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40696a-729b-4d43-9625-47b551c64e42 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:25.567543 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41262cb0-f5e3-4167-86c2-a1d90b79b605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:25.569187 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41262cb0-f5e3-4167-86c2-a1d90b79b605 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:25.569465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41262cb0-f5e3-4167-86c2-a1d90b79b605 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:25.569738 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41262cb0-f5e3-4167-86c2-a1d90b79b605 None admin] PATH_INFO: `/v3/users/791cf2b89171496ba5e2bb26f9f2c217` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:25.572580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e90e19-d8b7-4491-acc8-0d8df72f5b57 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:25.575749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41262cb0-f5e3-4167-86c2-a1d90b79b605 None admin] RBAC: Authorizing `identity:get_user(user_id=791cf2b89171496ba5e2bb26f9f2c217)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:25.577803 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41262cb0-f5e3-4167-86c2-a1d90b79b605 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:25.579762 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 148/290] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:25 2022] GET /identity/v3/users/791cf2b89171496ba5e2bb26f9f2c217 => generated 297 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:25.592010 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41262cb0-f5e3-4167-86c2-a1d90b79b605 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:25.599709 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:25.621469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d39cbb-e5e2-45c4-a6a8-39834630bec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:25.623023 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d39cbb-e5e2-45c4-a6a8-39834630bec9 None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:25.624324 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d39cbb-e5e2-45c4-a6a8-39834630bec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:25.624805 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d39cbb-e5e2-45c4-a6a8-39834630bec9 None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5/users/8ecee80fab8a4e8db799343ce838839a/roles/a7d155dd59914d22a32012302fa6d2d2` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:25.627176 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d39cbb-e5e2-45c4-a6a8-39834630bec9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5, user_id=8ecee80fab8a4e8db799343ce838839a, role_id=a7d155dd59914d22a32012302fa6d2d2)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:25.628900 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d39cbb-e5e2-45c4-a6a8-39834630bec9 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:25.650354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 143/291] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:20:25 2022] PUT /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5/users/8ecee80fab8a4e8db799343ce838839a/roles/a7d155dd59914d22a32012302fa6d2d2 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:20:25.725248 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:25.754831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41262cb0-f5e3-4167-86c2-a1d90b79b605 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:25.815781 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf2d2e-8805-42fd-8dd1-4daefb0b82d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:25.816478 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf2d2e-8805-42fd-8dd1-4daefb0b82d1 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:25.816774 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf2d2e-8805-42fd-8dd1-4daefb0b82d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:25.817041 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf2d2e-8805-42fd-8dd1-4daefb0b82d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:25.915441 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf2d2e-8805-42fd-8dd1-4daefb0b82d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:25.916915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf2d2e-8805-42fd-8dd1-4daefb0b82d1 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:26.003103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:26.027440 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 149/292] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:26.033359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d39cbb-e5e2-45c4-a6a8-39834630bec9 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:26.188485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d39cbb-e5e2-45c4-a6a8-39834630bec9 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:26.240531 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e10bfa-3c01-48ef-a0c6-33b98bd30f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:26.241407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e10bfa-3c01-48ef-a0c6-33b98bd30f5c None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:26.241687 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e10bfa-3c01-48ef-a0c6-33b98bd30f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:26.241961 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e10bfa-3c01-48ef-a0c6-33b98bd30f5c None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:26.243194 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e10bfa-3c01-48ef-a0c6-33b98bd30f5c None admin] RBAC: Authorizing `identity:get_project(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:26.244625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e10bfa-3c01-48ef-a0c6-33b98bd30f5c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:26.246344 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 144/293] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:26 2022] GET /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:26.253001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf2d2e-8805-42fd-8dd1-4daefb0b82d1 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:26.402789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf2d2e-8805-42fd-8dd1-4daefb0b82d1 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:26.439473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b2037-7085-4fe9-b45b-43760e52da05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:26.440135 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b2037-7085-4fe9-b45b-43760e52da05 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:26.440419 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b2037-7085-4fe9-b45b-43760e52da05 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:26.440701 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b2037-7085-4fe9-b45b-43760e52da05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:26.441546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b2037-7085-4fe9-b45b-43760e52da05 None admin] RBAC: Adding query filter params (role.id=b778e530d54948c7bc31f5b9231ea288, scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5, user.id=791cf2b89171496ba5e2bb26f9f2c217) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:26.441861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b2037-7085-4fe9-b45b-43760e52da05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:26.443334 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b2037-7085-4fe9-b45b-43760e52da05 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:26.450031 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 150/294] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:26 2022] GET /identity/v3/role_assignments?user.id=791cf2b89171496ba5e2bb26f9f2c217&scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5&role.id=b778e530d54948c7bc31f5b9231ea288 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:26.874046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9fdd3-1eff-430f-93bd-24f58515dbb6 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:26.874435 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9fdd3-1eff-430f-93bd-24f58515dbb6 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:26.874624 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9fdd3-1eff-430f-93bd-24f58515dbb6 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:26.876332 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 145/295] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:26.882006 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107af28-517e-455b-ba28-2c08394b35ad None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:26.882491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107af28-517e-455b-ba28-2c08394b35ad None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:26.882565 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107af28-517e-455b-ba28-2c08394b35ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:26.889626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996e5c0-e8fc-407a-ac8f-69e32b05f2ef None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:26.889934 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996e5c0-e8fc-407a-ac8f-69e32b05f2ef None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:26.890186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996e5c0-e8fc-407a-ac8f-69e32b05f2ef None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:26.891747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 146/296] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:26.897508 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc76a2f-b3b3-4ffe-a659-08727d674209 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:26.897797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc76a2f-b3b3-4ffe-a659-08727d674209 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:26.898073 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc76a2f-b3b3-4ffe-a659-08727d674209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:26.913211 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-0107af28-517e-455b-ba28-2c08394b35ad None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:26.914953 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0107af28-517e-455b-ba28-2c08394b35ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:26.925520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-7fc76a2f-b3b3-4ffe-a659-08727d674209 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:26.926750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-7fc76a2f-b3b3-4ffe-a659-08727d674209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:26.973150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:26.993000 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:27.004208 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 151/297] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:20:27.005883 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba95e4-40da-4465-9717-35849bf776c7 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:27.006371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba95e4-40da-4465-9717-35849bf776c7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:27.007308 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba95e4-40da-4465-9717-35849bf776c7 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:27.009101 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 152/298] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:27.016776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4af7e-4897-4b50-9129-f581da7405e2 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:27.017147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4af7e-4897-4b50-9129-f581da7405e2 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:27.017503 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4af7e-4897-4b50-9129-f581da7405e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:27.026309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 147/299] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:27.029614 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc76a2f-b3b3-4ffe-a659-08727d674209 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:27.040919 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-9cf4af7e-4897-4b50-9129-f581da7405e2 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:27.042375 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9cf4af7e-4897-4b50-9129-f581da7405e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:27.119054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 153/300] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:20:27.120431 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4af7e-4897-4b50-9129-f581da7405e2 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:27.142624 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc76a2f-b3b3-4ffe-a659-08727d674209 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:27.181088 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93969023-48df-4db1-b913-5beda3fd822e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:27.181897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93969023-48df-4db1-b913-5beda3fd822e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:27.182182 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93969023-48df-4db1-b913-5beda3fd822e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:27.182460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93969023-48df-4db1-b913-5beda3fd822e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:27.193519 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93969023-48df-4db1-b913-5beda3fd822e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:27.201567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-93969023-48df-4db1-b913-5beda3fd822e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 20:20:27.201567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:27.201567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:27.201567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:27.201567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:27.201567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:27.201567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:27.201567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:27.201567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:27.201567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:27.201567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:27.201567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:27.201567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:27.201567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:27.201567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:27.201567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:27.202600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:27.202600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:27.202600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:27.202600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:27.202600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:27.202600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:27.202600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:27.202600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:27.202600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:27.202600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:27.202600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:27.202600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:27.202600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:27.202600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:27.202600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:27.202600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:27.203433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:27.203433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:27.203433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:27.203433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:27.203433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:27.203433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:27.203433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:27.203433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:27.203433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:27.203433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:27.203433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 20:20:27.203433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:27.207968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 148/301] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:27.208940 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93969023-48df-4db1-b913-5beda3fd822e None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:27.236784 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4af7e-4897-4b50-9129-f581da7405e2 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:27.245211 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:27.251248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:27.276898 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6610b7d-f779-4ed9-b65a-2336a088e0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:27.278000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6610b7d-f779-4ed9-b65a-2336a088e0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:27.278287 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6610b7d-f779-4ed9-b65a-2336a088e0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:27.278565 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6610b7d-f779-4ed9-b65a-2336a088e0e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:27.280659 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6610b7d-f779-4ed9-b65a-2336a088e0e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:27.282144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6610b7d-f779-4ed9-b65a-2336a088e0e6 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:27.284217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 154/302] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:27.287081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6610b7d-f779-4ed9-b65a-2336a088e0e6 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:27.330558 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93969023-48df-4db1-b913-5beda3fd822e None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:27.368878 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d512f-90cb-444f-be4c-1ca90de38dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:27.369734 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d512f-90cb-444f-be4c-1ca90de38dcf None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:27.369972 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d512f-90cb-444f-be4c-1ca90de38dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:27.370253 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d512f-90cb-444f-be4c-1ca90de38dcf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:27.381460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d512f-90cb-444f-be4c-1ca90de38dcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:27.389282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-fa6d512f-90cb-444f-be4c-1ca90de38dcf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:27.389282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:27.389282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:27.389282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:27.389282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:27.389282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:27.389282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:27.389282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:27.389282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:27.389282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:27.389282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:27.389282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:27.389282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:27.389282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:27.389282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:27.389282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:27.390216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:27.390216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:27.390216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:27.390216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:27.390216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:27.390216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:27.390216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:27.390216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:27.390216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:27.390216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:27.390216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:27.390216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:27.390216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:27.390216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:27.390216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:27.390216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:27.391074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:27.391074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:27.391074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:27.391074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:27.391074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:27.391074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:27.391074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:27.391074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:27.391074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:27.391074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:27.391074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:27.391074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:27.391074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 149/303] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:27.392102 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d512f-90cb-444f-be4c-1ca90de38dcf None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:27.399646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6610b7d-f779-4ed9-b65a-2336a088e0e6 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:27.435177 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe87f5f-604c-4bb0-b931-ec62ab73481e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:27.435897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe87f5f-604c-4bb0-b931-ec62ab73481e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:27.436190 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe87f5f-604c-4bb0-b931-ec62ab73481e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:27.436471 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe87f5f-604c-4bb0-b931-ec62ab73481e None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:27.437233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe87f5f-604c-4bb0-b931-ec62ab73481e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:27.437543 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe87f5f-604c-4bb0-b931-ec62ab73481e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:27.438922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe87f5f-604c-4bb0-b931-ec62ab73481e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:27.447993 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 155/304] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:27.453817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe87f5f-604c-4bb0-b931-ec62ab73481e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:27.490130 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:27.494825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:27.501621 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d512f-90cb-444f-be4c-1ca90de38dcf None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:27.539065 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6240c-97a4-4d9d-8aa1-82a93fe766c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:27.539743 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6240c-97a4-4d9d-8aa1-82a93fe766c0 None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:27.540029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6240c-97a4-4d9d-8aa1-82a93fe766c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:27.540325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6240c-97a4-4d9d-8aa1-82a93fe766c0 None admin] PATH_INFO: `/v3/groups` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:27.541000 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6240c-97a4-4d9d-8aa1-82a93fe766c0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:27.542448 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6240c-97a4-4d9d-8aa1-82a93fe766c0 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:27.562368 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 150/305] 10.222.0.52 () {60 vars in 1134 bytes} [Thu Jul 14 20:20:27 2022] POST /identity/v3/groups => generated 229 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:20:27.562368 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6240c-97a4-4d9d-8aa1-82a93fe766c0 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:27.566988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe87f5f-604c-4bb0-b931-ec62ab73481e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:27.607851 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e37be-c684-4c6b-9cc5-da1e1590b735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:27.608575 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e37be-c684-4c6b-9cc5-da1e1590b735 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:27.608913 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e37be-c684-4c6b-9cc5-da1e1590b735 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:27.609250 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e37be-c684-4c6b-9cc5-da1e1590b735 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:27.610029 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e37be-c684-4c6b-9cc5-da1e1590b735 None admin] RBAC: Adding query filter params (name=member) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:27.610335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e37be-c684-4c6b-9cc5-da1e1590b735 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:27.611774 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e37be-c684-4c6b-9cc5-da1e1590b735 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:27.622146 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 156/306] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:27.628084 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e37be-c684-4c6b-9cc5-da1e1590b735 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:27.713917 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6240c-97a4-4d9d-8aa1-82a93fe766c0 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:27.761881 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c357802-2fec-4534-a6f1-794b66ed0f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:27.763166 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c357802-2fec-4534-a6f1-794b66ed0f05 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:27.763646 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c357802-2fec-4534-a6f1-794b66ed0f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:27.764679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c357802-2fec-4534-a6f1-794b66ed0f05 None admin] PATH_INFO: `/v3/users/ea7ca5941bd2450fa6784ece7e864dc7` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:27.767792 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:27.777794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e37be-c684-4c6b-9cc5-da1e1590b735 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:27.786261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c357802-2fec-4534-a6f1-794b66ed0f05 None admin] RBAC: Authorizing `identity:get_user(user_id=ea7ca5941bd2450fa6784ece7e864dc7)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:27.787432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c357802-2fec-4534-a6f1-794b66ed0f05 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:27.789284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 151/307] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:27 2022] GET /identity/v3/users/ea7ca5941bd2450fa6784ece7e864dc7 => generated 293 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:27.796703 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c357802-2fec-4534-a6f1-794b66ed0f05 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:27.813676 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:27.837448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fca48-6e11-4c5d-9f3d-558a4a93ccce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:27.838330 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fca48-6e11-4c5d-9f3d-558a4a93ccce None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:27.838772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fca48-6e11-4c5d-9f3d-558a4a93ccce None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:27.839092 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fca48-6e11-4c5d-9f3d-558a4a93ccce None admin] PATH_INFO: `/v3/users/c345adca638d46248bcc4415a55f6f76` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:27.841134 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8fca48-6e11-4c5d-9f3d-558a4a93ccce None admin] RBAC: Authorizing `identity:get_user(user_id=c345adca638d46248bcc4415a55f6f76)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:27.842628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8fca48-6e11-4c5d-9f3d-558a4a93ccce None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:27.844504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 157/308] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:27 2022] GET /identity/v3/users/c345adca638d46248bcc4415a55f6f76 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:27.849912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fca48-6e11-4c5d-9f3d-558a4a93ccce None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:27.920279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c357802-2fec-4534-a6f1-794b66ed0f05 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:27.968067 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391396b-ceb3-4250-9fec-2e8cfbc189ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:27.968706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391396b-ceb3-4250-9fec-2e8cfbc189ba None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:27.969298 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391396b-ceb3-4250-9fec-2e8cfbc189ba None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:27.969589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391396b-ceb3-4250-9fec-2e8cfbc189ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:27.970887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391396b-ceb3-4250-9fec-2e8cfbc189ba None admin] RBAC: Adding query filter params (role.id=a7d155dd59914d22a32012302fa6d2d2, user.id=ea7ca5941bd2450fa6784ece7e864dc7, scope.system=all) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:27.971230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391396b-ceb3-4250-9fec-2e8cfbc189ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:27.973250 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391396b-ceb3-4250-9fec-2e8cfbc189ba None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:27.980337 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fca48-6e11-4c5d-9f3d-558a4a93ccce None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:27.981027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 152/309] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Jul 14 20:20:27 2022] GET /identity/v3/role_assignments?user.id=ea7ca5941bd2450fa6784ece7e864dc7&scope.system=all&role.id=a7d155dd59914d22a32012302fa6d2d2 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:28.035796 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b78ba3-f93c-414f-9588-a16430ba538a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:28.037006 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b78ba3-f93c-414f-9588-a16430ba538a None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:28.037463 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b78ba3-f93c-414f-9588-a16430ba538a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:28.037878 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b78ba3-f93c-414f-9588-a16430ba538a None admin] PATH_INFO: `/v3/system/users/c345adca638d46248bcc4415a55f6f76/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:28.039844 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b78ba3-f93c-414f-9588-a16430ba538a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c345adca638d46248bcc4415a55f6f76, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:28.041511 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b78ba3-f93c-414f-9588-a16430ba538a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:28.054465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 158/310] 10.222.0.52 () {58 vars in 1334 bytes} [Thu Jul 14 20:20:27 2022] PUT /identity/v3/system/users/c345adca638d46248bcc4415a55f6f76/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:20:28.133517 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5ff1f-756d-4113-9cbf-7b94ce0e4f07 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:28.133815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5ff1f-756d-4113-9cbf-7b94ce0e4f07 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:28.134098 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5ff1f-756d-4113-9cbf-7b94ce0e4f07 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:28.135944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 153/311] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:28.142411 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007747b7-c93c-440d-bb88-0b661dfe85fb None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:28.143133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007747b7-c93c-440d-bb88-0b661dfe85fb None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:28.143883 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007747b7-c93c-440d-bb88-0b661dfe85fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:28.162792 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-007747b7-c93c-440d-bb88-0b661dfe85fb None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:28.164144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-007747b7-c93c-440d-bb88-0b661dfe85fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:28.198747 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:28.248189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 159/312] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:28.271357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5ff1f-756d-4113-9cbf-7b94ce0e4f07 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:28.359464 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:28.388957 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5ff1f-756d-4113-9cbf-7b94ce0e4f07 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:28.433221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ff568-1fb7-44b7-aa51-9c71abe12f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:28.434035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ff568-1fb7-44b7-aa51-9c71abe12f4c None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:28.434322 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ff568-1fb7-44b7-aa51-9c71abe12f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:28.434604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ff568-1fb7-44b7-aa51-9c71abe12f4c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:28.446578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ff568-1fb7-44b7-aa51-9c71abe12f4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:28.454494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-ef6ff568-1fb7-44b7-aa51-9c71abe12f4c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 20:20:28.454494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:28.454494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:28.454494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:28.454494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:28.454494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:28.454494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:28.454494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:28.454494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:28.454494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:28.454494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:28.454494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:28.454494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:28.454494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:28.454494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:28.454494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:28.455430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:28.455430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:28.455430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:28.455430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:28.455430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:28.455430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:28.455430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:28.455430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:28.455430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:28.455430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:28.455430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:28.455430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:28.455430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:28.455430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:28.455430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:28.455430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:28.456212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:28.456212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:28.456212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:28.456212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:28.456212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:28.456212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:28.456212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:28.456212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:28.456212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:28.456212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:28.456212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 20:20:28.456212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:28.456212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 154/313] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:28.464126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007747b7-c93c-440d-bb88-0b661dfe85fb None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:28.578192 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007747b7-c93c-440d-bb88-0b661dfe85fb None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:28.614477 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ee5cf-34ec-4640-bc81-e7d99cdb5182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:28.615184 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ee5cf-34ec-4640-bc81-e7d99cdb5182 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:28.615471 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ee5cf-34ec-4640-bc81-e7d99cdb5182 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:28.615748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ee5cf-34ec-4640-bc81-e7d99cdb5182 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:28.616509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ee5cf-34ec-4640-bc81-e7d99cdb5182 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:28.616818 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ee5cf-34ec-4640-bc81-e7d99cdb5182 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:28.618211 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ee5cf-34ec-4640-bc81-e7d99cdb5182 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:28.627126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 160/314] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:28.634892 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ff568-1fb7-44b7-aa51-9c71abe12f4c None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:28.746152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ff568-1fb7-44b7-aa51-9c71abe12f4c None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:28.792746 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9e663-1ca6-49b0-9bd4-c482b3a74e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:28.793662 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9e663-1ca6-49b0-9bd4-c482b3a74e11 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:28.793949 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9e663-1ca6-49b0-9bd4-c482b3a74e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:28.794224 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9e663-1ca6-49b0-9bd4-c482b3a74e11 None admin] PATH_INFO: `/v3/users/8ecee80fab8a4e8db799343ce838839a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:28.796883 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9e663-1ca6-49b0-9bd4-c482b3a74e11 None admin] RBAC: Authorizing `identity:get_user(user_id=8ecee80fab8a4e8db799343ce838839a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:28.798347 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9e663-1ca6-49b0-9bd4-c482b3a74e11 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:28.800893 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 155/315] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:28 2022] GET /identity/v3/users/8ecee80fab8a4e8db799343ce838839a => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:28.807498 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ee5cf-34ec-4640-bc81-e7d99cdb5182 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:28.835390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:28.921627 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ee5cf-34ec-4640-bc81-e7d99cdb5182 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:28.957348 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8aab36-9970-4a09-8e28-0f22f7336870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:28.958066 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8aab36-9970-4a09-8e28-0f22f7336870 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:28.958356 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8aab36-9970-4a09-8e28-0f22f7336870 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:28.958634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8aab36-9970-4a09-8e28-0f22f7336870 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:29.028374 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8aab36-9970-4a09-8e28-0f22f7336870 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:29.030382 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8aab36-9970-4a09-8e28-0f22f7336870 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:29.054335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:29.143093 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 161/316] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:29.149115 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9e663-1ca6-49b0-9bd4-c482b3a74e11 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:29.159722 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:29.258591 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9e663-1ca6-49b0-9bd4-c482b3a74e11 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:29.298616 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c129b79-6242-4157-88ad-2ff2d664650e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:29.299533 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c129b79-6242-4157-88ad-2ff2d664650e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:29.299820 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c129b79-6242-4157-88ad-2ff2d664650e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:29.300097 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c129b79-6242-4157-88ad-2ff2d664650e None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:29.301428 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c129b79-6242-4157-88ad-2ff2d664650e None admin] RBAC: Authorizing `identity:get_project(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:29.302995 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c129b79-6242-4157-88ad-2ff2d664650e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:29.309713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 156/317] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:29 2022] GET /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:29.311681 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8aab36-9970-4a09-8e28-0f22f7336870 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:29.423914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8aab36-9970-4a09-8e28-0f22f7336870 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:29.447822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:29.460274 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913150a-bd51-4a4f-ac28-a826027c034c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:29.460940 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913150a-bd51-4a4f-ac28-a826027c034c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:29.461226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913150a-bd51-4a4f-ac28-a826027c034c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:29.461508 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913150a-bd51-4a4f-ac28-a826027c034c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:29.462344 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913150a-bd51-4a4f-ac28-a826027c034c None admin] RBAC: Adding query filter params (role.id=a7d155dd59914d22a32012302fa6d2d2, scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5, user.id=8ecee80fab8a4e8db799343ce838839a) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:29.462669 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913150a-bd51-4a4f-ac28-a826027c034c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:29.464157 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913150a-bd51-4a4f-ac28-a826027c034c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:29.475062 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 162/318] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:29 2022] GET /identity/v3/role_assignments?user.id=8ecee80fab8a4e8db799343ce838839a&scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5&role.id=a7d155dd59914d22a32012302fa6d2d2 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:29.803317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc200c0-a04e-4de7-86b9-b93799e10095 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:29.803686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc200c0-a04e-4de7-86b9-b93799e10095 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:29.803884 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc200c0-a04e-4de7-86b9-b93799e10095 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:29.805399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 157/319] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:29.810424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fd4ae-788d-4d8a-adc5-084d0dd7f38e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:29.810716 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fd4ae-788d-4d8a-adc5-084d0dd7f38e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:29.811601 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fd4ae-788d-4d8a-adc5-084d0dd7f38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:29.829044 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-287fd4ae-788d-4d8a-adc5-084d0dd7f38e None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:29.830223 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-287fd4ae-788d-4d8a-adc5-084d0dd7f38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:29.903607 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 163/320] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:29.926115 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc200c0-a04e-4de7-86b9-b93799e10095 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:30.012461 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:30.039734 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc200c0-a04e-4de7-86b9-b93799e10095 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:30.075055 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89034c34-69cd-4ab1-804c-639f1288f4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:30.075785 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89034c34-69cd-4ab1-804c-639f1288f4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:30.076066 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89034c34-69cd-4ab1-804c-639f1288f4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:30.076341 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89034c34-69cd-4ab1-804c-639f1288f4e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:30.077625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89034c34-69cd-4ab1-804c-639f1288f4e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:30.079100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89034c34-69cd-4ab1-804c-639f1288f4e5 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:30.081239 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 158/321] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:30.086494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287fd4ae-788d-4d8a-adc5-084d0dd7f38e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:30.135212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987fb6e-f325-4911-aadb-8fb173c5e3f4 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:30.135527 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987fb6e-f325-4911-aadb-8fb173c5e3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:30.135819 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987fb6e-f325-4911-aadb-8fb173c5e3f4 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:30.137398 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 159/322] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:20:30.142427 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d997b7-a7f1-4919-8250-51b9f7cfcacb None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:30.142721 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d997b7-a7f1-4919-8250-51b9f7cfcacb None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:30.143048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d997b7-a7f1-4919-8250-51b9f7cfcacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:30.162151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-26d997b7-a7f1-4919-8250-51b9f7cfcacb None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:30.163439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-26d997b7-a7f1-4919-8250-51b9f7cfcacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:30.192322 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287fd4ae-788d-4d8a-adc5-084d0dd7f38e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:30.212041 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:30.231343 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbbbb8-0fe4-40ab-b13e-d86d695e8c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:30.232016 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbbbb8-0fe4-40ab-b13e-d86d695e8c60 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:30.232297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbbbb8-0fe4-40ab-b13e-d86d695e8c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:30.232574 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbbbb8-0fe4-40ab-b13e-d86d695e8c60 None admin] PATH_INFO: `/v3/groups` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:30.233825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbbbb8-0fe4-40ab-b13e-d86d695e8c60 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:30.234951 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 160/323] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:20:30.235289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbbbb8-0fe4-40ab-b13e-d86d695e8c60 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:30.239539 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba200845-c729-4710-9ea4-d66ce660f7a1 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:30.240394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba200845-c729-4710-9ea4-d66ce660f7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:30.240588 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba200845-c729-4710-9ea4-d66ce660f7a1 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:30.242687 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 161/324] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:30.248509 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19390639-45d6-458b-bda1-e88b7bfa1d1a None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:30.248800 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19390639-45d6-458b-bda1-e88b7bfa1d1a None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:30.249077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19390639-45d6-458b-bda1-e88b7bfa1d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:30.251725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 164/325] 10.222.0.52 () {60 vars in 1134 bytes} [Thu Jul 14 20:20:30 2022] POST /identity/v3/groups => generated 226 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:20:30.257939 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbbbb8-0fe4-40ab-b13e-d86d695e8c60 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:30.271537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-19390639-45d6-458b-bda1-e88b7bfa1d1a None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:30.272787 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-19390639-45d6-458b-bda1-e88b7bfa1d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:30.350570 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 162/326] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:20:30.374339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19390639-45d6-458b-bda1-e88b7bfa1d1a None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:30.376752 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:30.380830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbbbb8-0fe4-40ab-b13e-d86d695e8c60 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:30.422462 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db0cab-56df-448f-8d25-227fc5d2d92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:30.423668 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db0cab-56df-448f-8d25-227fc5d2d92a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:30.424442 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db0cab-56df-448f-8d25-227fc5d2d92a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:30.424865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db0cab-56df-448f-8d25-227fc5d2d92a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:30.439158 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db0cab-56df-448f-8d25-227fc5d2d92a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:30.448033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-37db0cab-56df-448f-8d25-227fc5d2d92a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 20:20:30.448033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:30.448033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:30.448033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:30.448033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:30.448033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:30.448033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:30.448033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:30.448033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:30.448033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:30.448033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:30.448033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:30.448033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:30.448033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:30.448033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:30.448033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:30.449118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:30.449118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:30.449118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:30.449118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:30.449118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:30.449118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:30.449118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:30.449118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:30.449118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:30.449118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:30.449118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:30.449118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:30.449118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:30.449118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:30.449118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:30.449118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:30.449985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:30.449985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:30.449985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:30.449985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:30.449985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:30.449985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:30.449985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:30.449985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:30.449985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:30.449985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:30.449985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 20:20:30.449985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:30.454866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 165/327] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:30.455881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db0cab-56df-448f-8d25-227fc5d2d92a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:30.502289 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19390639-45d6-458b-bda1-e88b7bfa1d1a None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:30.507639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:30.545355 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da65e98-827f-4aab-b8b7-2af86b677950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:30.546239 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da65e98-827f-4aab-b8b7-2af86b677950 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:30.546522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da65e98-827f-4aab-b8b7-2af86b677950 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:30.547135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da65e98-827f-4aab-b8b7-2af86b677950 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:30.561343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da65e98-827f-4aab-b8b7-2af86b677950 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:30.569889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-9da65e98-827f-4aab-b8b7-2af86b677950 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:30.569889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:30.569889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:30.569889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:30.569889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:30.569889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:30.569889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:30.569889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:30.569889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:30.569889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:30.569889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:30.569889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:30.569889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:30.569889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:30.569889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:30.569889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:30.571154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:30.571154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:30.571154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:30.571154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:30.571154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:30.571154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:30.571154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:30.571154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:30.571154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:30.571154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:30.571154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:30.571154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:30.571154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:30.571154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:30.571154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:30.571154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:30.572471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:30.572471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:30.572471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:30.572471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:30.572471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:30.572471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:30.572471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:30.572471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:30.572471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:30.572471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:30.572471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:30.572471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:30.572471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 163/328] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:30.578171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da65e98-827f-4aab-b8b7-2af86b677950 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:30.588295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db0cab-56df-448f-8d25-227fc5d2d92a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:30.632036 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027f453-9404-487a-8e9c-96a2cb990c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:30.633188 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027f453-9404-487a-8e9c-96a2cb990c9c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:30.633188 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027f453-9404-487a-8e9c-96a2cb990c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:30.633568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027f453-9404-487a-8e9c-96a2cb990c9c None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:30.634416 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027f453-9404-487a-8e9c-96a2cb990c9c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:30.634814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027f453-9404-487a-8e9c-96a2cb990c9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:30.636346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027f453-9404-487a-8e9c-96a2cb990c9c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:30.648826 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 166/329] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:30.656102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027f453-9404-487a-8e9c-96a2cb990c9c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:30.678905 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:30.739623 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da65e98-827f-4aab-b8b7-2af86b677950 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:30.787648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07699914-80df-4b26-8204-1a329dee28ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:30.788387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07699914-80df-4b26-8204-1a329dee28ab None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:30.788672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07699914-80df-4b26-8204-1a329dee28ab None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:30.788948 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07699914-80df-4b26-8204-1a329dee28ab None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:30.789716 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07699914-80df-4b26-8204-1a329dee28ab None admin] RBAC: Adding query filter params (name=member) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:30.790021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07699914-80df-4b26-8204-1a329dee28ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:30.792044 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07699914-80df-4b26-8204-1a329dee28ab None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:30.802454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:30.803455 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 164/330] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:30.808969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027f453-9404-487a-8e9c-96a2cb990c9c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:30.809804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07699914-80df-4b26-8204-1a329dee28ab None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:30.857630 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b82dec-422a-4dae-8d4a-1a2945df4a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:30.858907 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b82dec-422a-4dae-8d4a-1a2945df4a3c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:30.859628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b82dec-422a-4dae-8d4a-1a2945df4a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:30.861059 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b82dec-422a-4dae-8d4a-1a2945df4a3c None admin] PATH_INFO: `/v3/users/ea7ca5941bd2450fa6784ece7e864dc7` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:30.862567 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b82dec-422a-4dae-8d4a-1a2945df4a3c None admin] RBAC: Authorizing `identity:get_user(user_id=ea7ca5941bd2450fa6784ece7e864dc7)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:30.864288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b82dec-422a-4dae-8d4a-1a2945df4a3c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:30.867113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 167/331] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:30 2022] GET /identity/v3/users/ea7ca5941bd2450fa6784ece7e864dc7 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:30.873105 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b82dec-422a-4dae-8d4a-1a2945df4a3c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:30.955715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07699914-80df-4b26-8204-1a329dee28ab None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:30.990768 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:31.005766 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a2263-6243-4bea-b8e1-23dabc90a8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:31.007025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a2263-6243-4bea-b8e1-23dabc90a8e9 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:31.007518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a2263-6243-4bea-b8e1-23dabc90a8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:31.008027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a2263-6243-4bea-b8e1-23dabc90a8e9 None admin] PATH_INFO: `/v3/users/c345adca638d46248bcc4415a55f6f76` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:31.009522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a2263-6243-4bea-b8e1-23dabc90a8e9 None admin] RBAC: Authorizing `identity:get_user(user_id=c345adca638d46248bcc4415a55f6f76)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:31.012319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a2263-6243-4bea-b8e1-23dabc90a8e9 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:31.014862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 165/332] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:30 2022] GET /identity/v3/users/c345adca638d46248bcc4415a55f6f76 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:31.022559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a2263-6243-4bea-b8e1-23dabc90a8e9 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:31.027440 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b82dec-422a-4dae-8d4a-1a2945df4a3c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:31.075319 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b0b51-00bb-41f4-9d6e-e37364a764ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:31.077185 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b0b51-00bb-41f4-9d6e-e37364a764ab None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:31.077673 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b0b51-00bb-41f4-9d6e-e37364a764ab None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:31.078156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b0b51-00bb-41f4-9d6e-e37364a764ab None admin] PATH_INFO: `/v3/system/users/ea7ca5941bd2450fa6784ece7e864dc7/roles/a7d155dd59914d22a32012302fa6d2d2` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:31.081310 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b0b51-00bb-41f4-9d6e-e37364a764ab None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ea7ca5941bd2450fa6784ece7e864dc7, role_id=a7d155dd59914d22a32012302fa6d2d2)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:31.083100 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b0b51-00bb-41f4-9d6e-e37364a764ab None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:31.095377 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 168/333] 10.222.0.52 () {58 vars in 1334 bytes} [Thu Jul 14 20:20:30 2022] PUT /identity/v3/system/users/ea7ca5941bd2450fa6784ece7e864dc7/roles/a7d155dd59914d22a32012302fa6d2d2 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:20:31.168556 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a2263-6243-4bea-b8e1-23dabc90a8e9 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:31.211242 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ccdb14-6dde-461d-b1c2-27dfb5eea3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:31.211844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccdb14-6dde-461d-b1c2-27dfb5eea3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:31.212136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccdb14-6dde-461d-b1c2-27dfb5eea3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:31.212414 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccdb14-6dde-461d-b1c2-27dfb5eea3a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:31.213325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ccdb14-6dde-461d-b1c2-27dfb5eea3a5 None admin] RBAC: Adding query filter params (role.id=b778e530d54948c7bc31f5b9231ea288, user.id=c345adca638d46248bcc4415a55f6f76, scope.system=all) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:31.213694 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ccdb14-6dde-461d-b1c2-27dfb5eea3a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:31.215191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ccdb14-6dde-461d-b1c2-27dfb5eea3a5 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:31.230000 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 166/334] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Jul 14 20:20:31 2022] GET /identity/v3/role_assignments?user.id=c345adca638d46248bcc4415a55f6f76&scope.system=all&role.id=b778e530d54948c7bc31f5b9231ea288 => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:32.634744 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7a395-bdbe-4097-a12e-d92f4ea129f1 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:32.635674 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7a395-bdbe-4097-a12e-d92f4ea129f1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:32.636156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7a395-bdbe-4097-a12e-d92f4ea129f1 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:32.638012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 169/335] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:32.644768 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3b9fc-8a57-4013-b1d1-d1c7c0e917bc None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:32.645657 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3b9fc-8a57-4013-b1d1-d1c7c0e917bc None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:32.646137 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3b9fc-8a57-4013-b1d1-d1c7c0e917bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:32.669090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-1db3b9fc-8a57-4013-b1d1-d1c7c0e917bc None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:32.671178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-1db3b9fc-8a57-4013-b1d1-d1c7c0e917bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:32.742058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:32.765220 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 167/336] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:20:32.813042 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7a395-bdbe-4097-a12e-d92f4ea129f1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:32.846778 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:32.947743 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7a395-bdbe-4097-a12e-d92f4ea129f1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:32.985418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce64f86-519d-4946-abca-f1d543e57ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:32.986316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce64f86-519d-4946-abca-f1d543e57ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:32.986597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce64f86-519d-4946-abca-f1d543e57ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:32.986873 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce64f86-519d-4946-abca-f1d543e57ba4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:32.998985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce64f86-519d-4946-abca-f1d543e57ba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:33.006963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-4ce64f86-519d-4946-abca-f1d543e57ba4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:33.006963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:33.006963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:33.006963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:33.006963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:33.006963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:33.006963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:33.006963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:33.006963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:33.006963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:33.006963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:33.006963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:33.006963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:33.006963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:33.006963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:33.006963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:33.007847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:33.007847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:33.007847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:33.007847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:33.007847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:33.007847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:33.007847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:33.007847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:33.007847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:33.007847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:33.007847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:33.007847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:33.007847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:33.007847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:33.007847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:33.007847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:33.008621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:33.008621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:33.008621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:33.008621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:33.008621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:33.008621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:33.008621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:33.008621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:33.008621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:33.008621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:33.008621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:33.008621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:33.013250 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 170/337] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:33.014670 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3b9fc-8a57-4013-b1d1-d1c7c0e917bc None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:33.145845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3b9fc-8a57-4013-b1d1-d1c7c0e917bc None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:33.197139 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e812bae-59fe-42cd-8982-46e3e5977623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:33.197888 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e812bae-59fe-42cd-8982-46e3e5977623 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:33.198121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e812bae-59fe-42cd-8982-46e3e5977623 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:33.198404 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e812bae-59fe-42cd-8982-46e3e5977623 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:33.199198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e812bae-59fe-42cd-8982-46e3e5977623 None admin] RBAC: Adding query filter params (name=member) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:33.199510 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e812bae-59fe-42cd-8982-46e3e5977623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:33.200919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e812bae-59fe-42cd-8982-46e3e5977623 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:33.214989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 168/338] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:33.221577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce64f86-519d-4946-abca-f1d543e57ba4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:33.328928 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:33.347093 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce64f86-519d-4946-abca-f1d543e57ba4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:33.352870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87ba5c-dc27-4fff-ad91-1959102f29e0 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:33.353350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87ba5c-dc27-4fff-ad91-1959102f29e0 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:33.353775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87ba5c-dc27-4fff-ad91-1959102f29e0 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:33.355371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 169/339] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:20:33.364385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0e595-b4ab-4f2f-b45c-573c7c3f2c78 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:33.364804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0e595-b4ab-4f2f-b45c-573c7c3f2c78 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:33.365180 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0e595-b4ab-4f2f-b45c-573c7c3f2c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:33.389977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f34393-da06-4ac4-aa45-f4387bb4476c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:33.391082 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f34393-da06-4ac4-aa45-f4387bb4476c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:33.391519 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f34393-da06-4ac4-aa45-f4387bb4476c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:33.392012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f34393-da06-4ac4-aa45-f4387bb4476c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:33.404114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-82c0e595-b4ab-4f2f-b45c-573c7c3f2c78 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:33.405603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-82c0e595-b4ab-4f2f-b45c-573c7c3f2c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:33.471724 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f34393-da06-4ac4-aa45-f4387bb4476c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:33.473470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f34393-da06-4ac4-aa45-f4387bb4476c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:33.516063 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:33.572109 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:33.586818 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 171/340] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:33.588658 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f34393-da06-4ac4-aa45-f4387bb4476c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:33.599815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 170/341] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:33.644301 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0e595-b4ab-4f2f-b45c-573c7c3f2c78 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:33.703139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f34393-da06-4ac4-aa45-f4387bb4476c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:33.750436 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b23399-b9f5-4f40-a945-2053805d008e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:33.751701 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b23399-b9f5-4f40-a945-2053805d008e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:33.752205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b23399-b9f5-4f40-a945-2053805d008e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:33.752680 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b23399-b9f5-4f40-a945-2053805d008e None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:33.754123 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b23399-b9f5-4f40-a945-2053805d008e None admin] RBAC: Authorizing `identity:get_project(project_id=930b378be44b4bb7a2009c72a424f1d6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:33.759934 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b23399-b9f5-4f40-a945-2053805d008e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:33.763284 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 172/342] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:33 2022] GET /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:33.767981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b23399-b9f5-4f40-a945-2053805d008e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:33.816217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0e595-b4ab-4f2f-b45c-573c7c3f2c78 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:33.861245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:33.880487 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddaf8f-eea6-4386-b330-ee9affa3ae1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:33.881658 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddaf8f-eea6-4386-b330-ee9affa3ae1d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:33.883781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddaf8f-eea6-4386-b330-ee9affa3ae1d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:33.889103 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddaf8f-eea6-4386-b330-ee9affa3ae1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:33.902498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddaf8f-eea6-4386-b330-ee9affa3ae1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:33.915292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-4dddaf8f-eea6-4386-b330-ee9affa3ae1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:33.915292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:33.915292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:33.915292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:33.915292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:33.915292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:33.915292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:33.915292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:33.915292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:33.915292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:33.915292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:33.915292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:33.915292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:33.915292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:33.915292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:33.915292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:33.916240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:33.916240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:33.916240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:33.916240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:33.916240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:33.916240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:33.916240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:33.916240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:33.916240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:33.916240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:33.916240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:33.916240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:33.916240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:33.916240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:33.916240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:33.916240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:33.917043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:33.917043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:33.917043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:33.917043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:33.917043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:33.917043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:33.917043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:33.917043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:33.917043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:33.917043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:33.917043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:33.917043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:33.921281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 171/343] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:20:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:33.923038 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bbf82-56d5-4c61-bac6-8752b60af2a4 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:33.923484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bbf82-56d5-4c61-bac6-8752b60af2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:33.923907 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bbf82-56d5-4c61-bac6-8752b60af2a4 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:33.925498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 172/344] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:33.930926 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda74bbe-acfc-4846-886e-07cc14863245 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:33.931475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda74bbe-acfc-4846-886e-07cc14863245 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:33.931608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda74bbe-acfc-4846-886e-07cc14863245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:33.941005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b23399-b9f5-4f40-a945-2053805d008e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:33.963052 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-cda74bbe-acfc-4846-886e-07cc14863245 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:33.964251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-cda74bbe-acfc-4846-886e-07cc14863245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:34.018800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ffc35-63ea-4b4d-bbbf-334e193a5656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:34.019298 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:34.019618 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ffc35-63ea-4b4d-bbbf-334e193a5656 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:34.019898 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ffc35-63ea-4b4d-bbbf-334e193a5656 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:34.020170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ffc35-63ea-4b4d-bbbf-334e193a5656 None admin] PATH_INFO: `/v3/groups/6e193bbfa6174cbdb7b4484137920fb2` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:34.028719 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ffc35-63ea-4b4d-bbbf-334e193a5656 None admin] RBAC: Authorizing `identity:get_group(group_id=6e193bbfa6174cbdb7b4484137920fb2)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:34.030710 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ffc35-63ea-4b4d-bbbf-334e193a5656 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:34.033103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 173/345] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 14 20:20:33 2022] GET /identity/v3/groups/6e193bbfa6174cbdb7b4484137920fb2 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:34.033886 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ffc35-63ea-4b4d-bbbf-334e193a5656 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:34.109399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 173/346] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:20:34.110280 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda74bbe-acfc-4846-886e-07cc14863245 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:34.208726 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ffc35-63ea-4b4d-bbbf-334e193a5656 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:34.245372 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f040f-139e-4a63-8f50-99419d3321db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:34.246048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f040f-139e-4a63-8f50-99419d3321db None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:34.246329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f040f-139e-4a63-8f50-99419d3321db None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:34.246602 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f040f-139e-4a63-8f50-99419d3321db None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:34.247242 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda74bbe-acfc-4846-886e-07cc14863245 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:34.247421 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f040f-139e-4a63-8f50-99419d3321db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:34.247692 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f040f-139e-4a63-8f50-99419d3321db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:34.249106 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f040f-139e-4a63-8f50-99419d3321db None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:34.260700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 174/347] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:20:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:34.261987 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5d110-5a65-4940-8167-3b727133f795 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:34.262277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5d110-5a65-4940-8167-3b727133f795 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:34.262568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5d110-5a65-4940-8167-3b727133f795 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:34.269515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 175/348] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:34.269515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5d110-5a65-4940-8167-3b727133f795 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:34.269515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:34.304729 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c206163-e7de-4a67-b046-83fdb624fbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:34.305645 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c206163-e7de-4a67-b046-83fdb624fbca None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:34.306072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c206163-e7de-4a67-b046-83fdb624fbca None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:34.306519 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c206163-e7de-4a67-b046-83fdb624fbca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:34.307582 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c206163-e7de-4a67-b046-83fdb624fbca None admin] RBAC: Adding query filter params (group.id=6e193bbfa6174cbdb7b4484137920fb2, role.id=b778e530d54948c7bc31f5b9231ea288, scope.project.id=930b378be44b4bb7a2009c72a424f1d6) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:34.308075 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c206163-e7de-4a67-b046-83fdb624fbca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:34.309659 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c206163-e7de-4a67-b046-83fdb624fbca None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:34.316399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 174/349] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 14 20:20:34 2022] GET /identity/v3/role_assignments?group.id=6e193bbfa6174cbdb7b4484137920fb2&scope.project.id=930b378be44b4bb7a2009c72a424f1d6&role.id=b778e530d54948c7bc31f5b9231ea288 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:34.317600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c206163-e7de-4a67-b046-83fdb624fbca None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:34.353388 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:34.417210 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5d110-5a65-4940-8167-3b727133f795 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:34.473764 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c206163-e7de-4a67-b046-83fdb624fbca None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:34.491111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab0608-ef59-4539-afd0-8d1ae1c6b79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:34.492217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab0608-ef59-4539-afd0-8d1ae1c6b79c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:34.492644 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab0608-ef59-4539-afd0-8d1ae1c6b79c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:34.493058 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab0608-ef59-4539-afd0-8d1ae1c6b79c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:34.515896 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab0608-ef59-4539-afd0-8d1ae1c6b79c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:34.524401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-40ab0608-ef59-4539-afd0-8d1ae1c6b79c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:34.524401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:34.524401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:34.524401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:34.524401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:34.524401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:34.524401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:34.524401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:34.524401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:34.524401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:34.524401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:34.524401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:34.524401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:34.524401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:34.524401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:34.524401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:34.525504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:34.525504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:34.525504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:34.525504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:34.525504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:34.525504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:34.525504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:34.525504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:34.525504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:34.525504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:34.525504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:34.525504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:34.525504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:34.525504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:34.525504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:34.525504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:34.526454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:34.526454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:34.526454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:34.526454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:34.526454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:34.526454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:34.526454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:34.526454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:34.526454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:34.526454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:34.526454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:34.526454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:34.528825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 176/350] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:34.534966 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8b87c-dfed-4c61-b85f-4cf83af23406 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:34.535729 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8b87c-dfed-4c61-b85f-4cf83af23406 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:34.536159 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8b87c-dfed-4c61-b85f-4cf83af23406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:34.559015 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bfc8c8-d0fe-4f87-997d-174cea9b70ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:34.559506 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-2aa8b87c-dfed-4c61-b85f-4cf83af23406 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:34.559724 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bfc8c8-d0fe-4f87-997d-174cea9b70ff None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:34.560014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bfc8c8-d0fe-4f87-997d-174cea9b70ff None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:34.560294 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bfc8c8-d0fe-4f87-997d-174cea9b70ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:34.564836 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2aa8b87c-dfed-4c61-b85f-4cf83af23406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:34.620170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:34.654453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 177/351] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:34.655602 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8b87c-dfed-4c61-b85f-4cf83af23406 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:34.686825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bfc8c8-d0fe-4f87-997d-174cea9b70ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:34.688443 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bfc8c8-d0fe-4f87-997d-174cea9b70ff None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:34.735002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:34.779896 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8b87c-dfed-4c61-b85f-4cf83af23406 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:34.817729 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf977d7-4760-4bfb-a207-a98179226c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:34.818697 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf977d7-4760-4bfb-a207-a98179226c6c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:34.819213 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf977d7-4760-4bfb-a207-a98179226c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:34.819621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf977d7-4760-4bfb-a207-a98179226c6c None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:34.820565 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf977d7-4760-4bfb-a207-a98179226c6c None admin] RBAC: Adding query filter params (name=member) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:34.821072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf977d7-4760-4bfb-a207-a98179226c6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:34.822618 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf977d7-4760-4bfb-a207-a98179226c6c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:34.831541 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 175/352] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:34.833010 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc853f-2828-4a56-89af-8aa008a2f408 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:34.833374 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc853f-2828-4a56-89af-8aa008a2f408 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:34.839637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc853f-2828-4a56-89af-8aa008a2f408 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:34.839637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 176/353] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:34.839637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc853f-2828-4a56-89af-8aa008a2f408 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:34.839897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 178/354] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:34.841514 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf977d7-4760-4bfb-a207-a98179226c6c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:34.899149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:34.987284 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf977d7-4760-4bfb-a207-a98179226c6c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:34.989419 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc853f-2828-4a56-89af-8aa008a2f408 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:35.026592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:35.037422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ce765-a646-4850-b735-53eeb6a23724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:35.038131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ce765-a646-4850-b735-53eeb6a23724 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:35.038439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ce765-a646-4850-b735-53eeb6a23724 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:35.038688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ce765-a646-4850-b735-53eeb6a23724 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:35.041839 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af139f-2d59-44e7-b301-367ca8223ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:35.045547 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af139f-2d59-44e7-b301-367ca8223ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:35.046125 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af139f-2d59-44e7-b301-367ca8223ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:35.046629 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af139f-2d59-44e7-b301-367ca8223ce0 None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:35.048663 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af139f-2d59-44e7-b301-367ca8223ce0 None admin] RBAC: Authorizing `identity:get_project(project_id=930b378be44b4bb7a2009c72a424f1d6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:35.050635 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af139f-2d59-44e7-b301-367ca8223ce0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:35.053099 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 179/355] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:34 2022] GET /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:35.054653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d80a6-e73f-487f-bf2c-e36965ee7143 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:35.055190 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d80a6-e73f-487f-bf2c-e36965ee7143 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:35.055657 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d80a6-e73f-487f-bf2c-e36965ee7143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:35.058573 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ce765-a646-4850-b735-53eeb6a23724 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:35.067731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-dd2ce765-a646-4850-b735-53eeb6a23724 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 20:20:35.067731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:35.067731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:35.067731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:35.067731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:35.067731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:35.067731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:35.067731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:35.067731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:35.067731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:35.067731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:35.067731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:35.067731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:35.067731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:35.067731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:35.067731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:35.068863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:35.068863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:35.068863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:35.068863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:35.068863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:35.068863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:35.068863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:35.068863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:35.068863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:35.068863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:35.068863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:35.068863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:35.068863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:35.068863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:35.068863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:35.068863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:35.069637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:35.069637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:35.069637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:35.069637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:35.069637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:35.069637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:35.069637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:35.069637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:35.069637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:35.069637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:35.069637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 20:20:35.069637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:35.069637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 177/356] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:35.070285 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ce765-a646-4850-b735-53eeb6a23724 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:35.081834 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-257d80a6-e73f-487f-bf2c-e36965ee7143 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:35.083395 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-257d80a6-e73f-487f-bf2c-e36965ee7143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:35.183393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 180/357] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:35.184861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d80a6-e73f-487f-bf2c-e36965ee7143 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:35.227749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:35.228992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ce765-a646-4850-b735-53eeb6a23724 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:35.272674 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bb8e7-4a2c-4f57-a17a-a3dee5714dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:35.273418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bb8e7-4a2c-4f57-a17a-a3dee5714dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:35.273725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bb8e7-4a2c-4f57-a17a-a3dee5714dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:35.274442 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bb8e7-4a2c-4f57-a17a-a3dee5714dc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:35.319022 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d80a6-e73f-487f-bf2c-e36965ee7143 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:35.345792 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:35.372175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bb8e7-4a2c-4f57-a17a-a3dee5714dc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:35.373869 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bb8e7-4a2c-4f57-a17a-a3dee5714dc6 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:35.384688 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9e41d-43ca-41c0-8424-b87b8cfaabab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:35.385574 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9e41d-43ca-41c0-8424-b87b8cfaabab None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:35.385856 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9e41d-43ca-41c0-8424-b87b8cfaabab None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:35.386128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9e41d-43ca-41c0-8424-b87b8cfaabab None admin] PATH_INFO: `/v3/groups/6e193bbfa6174cbdb7b4484137920fb2` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:35.389558 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9e41d-43ca-41c0-8424-b87b8cfaabab None admin] RBAC: Authorizing `identity:get_group(group_id=6e193bbfa6174cbdb7b4484137920fb2)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:35.390827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9e41d-43ca-41c0-8424-b87b8cfaabab None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:35.392832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 181/358] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 14 20:20:35 2022] GET /identity/v3/groups/6e193bbfa6174cbdb7b4484137920fb2 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:35.393926 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9e41d-43ca-41c0-8424-b87b8cfaabab None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:35.526532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 178/359] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:35.527359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bb8e7-4a2c-4f57-a17a-a3dee5714dc6 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:35.538436 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9e41d-43ca-41c0-8424-b87b8cfaabab None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:35.540537 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:35.580904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29295e-bf0f-4c36-80f1-59af5fe5123e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:35.581543 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29295e-bf0f-4c36-80f1-59af5fe5123e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:35.581834 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29295e-bf0f-4c36-80f1-59af5fe5123e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:35.582110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29295e-bf0f-4c36-80f1-59af5fe5123e None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:35.582873 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29295e-bf0f-4c36-80f1-59af5fe5123e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:35.584800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29295e-bf0f-4c36-80f1-59af5fe5123e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:35.585583 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29295e-bf0f-4c36-80f1-59af5fe5123e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:35.596661 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 182/360] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:35.597994 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314c153-673d-46e4-b581-3fce21ebe000 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:35.598286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314c153-673d-46e4-b581-3fce21ebe000 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:35.598565 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314c153-673d-46e4-b581-3fce21ebe000 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:35.599954 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 183/361] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:35.605608 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314c153-673d-46e4-b581-3fce21ebe000 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:35.655275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:35.682719 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bb8e7-4a2c-4f57-a17a-a3dee5714dc6 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:35.741589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d3ae4-3cb9-406c-b53d-d69bb22bf339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:35.742605 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d3ae4-3cb9-406c-b53d-d69bb22bf339 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:35.742997 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d3ae4-3cb9-406c-b53d-d69bb22bf339 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:35.743349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d3ae4-3cb9-406c-b53d-d69bb22bf339 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:35.758599 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d3ae4-3cb9-406c-b53d-d69bb22bf339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:35.760459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314c153-673d-46e4-b581-3fce21ebe000 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:35.770826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-a48d3ae4-3cb9-406c-b53d-d69bb22bf339 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:35.770826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:35.770826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:35.770826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:35.770826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:35.770826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:35.770826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:35.770826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:35.770826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:35.770826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:35.770826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:35.770826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:35.770826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:35.770826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:35.770826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:35.770826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:35.771832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:35.771832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:35.771832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:35.771832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:35.771832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:35.771832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:35.771832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:35.771832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:35.771832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:35.771832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:35.771832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:35.771832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:35.771832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:35.771832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:35.771832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:35.771832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:35.773217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:35.773217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:35.773217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:35.773217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:35.773217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:35.773217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:35.773217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:35.773217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:35.773217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:35.773217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:35.773217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:35.773217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:35.773217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 179/362] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:20:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:35.773987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d3ae4-3cb9-406c-b53d-d69bb22bf339 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:35.816663 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd98a74-22b8-4e9f-a8d2-0013fe9b0de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:35.817399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd98a74-22b8-4e9f-a8d2-0013fe9b0de6 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:35.817698 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd98a74-22b8-4e9f-a8d2-0013fe9b0de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:35.817855 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd98a74-22b8-4e9f-a8d2-0013fe9b0de6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:35.818801 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd98a74-22b8-4e9f-a8d2-0013fe9b0de6 None admin] RBAC: Adding query filter params (group.id=6e193bbfa6174cbdb7b4484137920fb2, role.id=6b2eb3e4cefd4a2db4f18e04a158600a, scope.project.id=930b378be44b4bb7a2009c72a424f1d6) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:35.820266 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd98a74-22b8-4e9f-a8d2-0013fe9b0de6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:35.820741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd98a74-22b8-4e9f-a8d2-0013fe9b0de6 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:35.829393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 184/363] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 14 20:20:35 2022] GET /identity/v3/role_assignments?group.id=6e193bbfa6174cbdb7b4484137920fb2&scope.project.id=930b378be44b4bb7a2009c72a424f1d6&role.id=6b2eb3e4cefd4a2db4f18e04a158600a => generated 260 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:35.831043 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dac8b6-1b16-4644-9751-1f9bec1829cc None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:35.831198 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dac8b6-1b16-4644-9751-1f9bec1829cc None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:35.831418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dac8b6-1b16-4644-9751-1f9bec1829cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:35.851224 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e6dac8b6-1b16-4644-9751-1f9bec1829cc None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:35.852587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e6dac8b6-1b16-4644-9751-1f9bec1829cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:35.879203 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:35.933049 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d3ae4-3cb9-406c-b53d-d69bb22bf339 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:35.984987 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 185/364] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:35.988104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb5689-ff3e-4620-bd9c-5fd5148bb723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:35.988104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb5689-ff3e-4620-bd9c-5fd5148bb723 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:35.988453 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb5689-ff3e-4620-bd9c-5fd5148bb723 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:35.988724 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb5689-ff3e-4620-bd9c-5fd5148bb723 None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:35.989961 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb5689-ff3e-4620-bd9c-5fd5148bb723 None admin] RBAC: Authorizing `identity:get_project(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:35.991069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dac8b6-1b16-4644-9751-1f9bec1829cc None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:35.991492 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb5689-ff3e-4620-bd9c-5fd5148bb723 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:35.993528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 180/365] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:35 2022] GET /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:35.994298 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb5689-ff3e-4620-bd9c-5fd5148bb723 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:36.018788 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:36.166064 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dac8b6-1b16-4644-9751-1f9bec1829cc None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:36.170166 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb5689-ff3e-4620-bd9c-5fd5148bb723 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:36.238756 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284822c-950f-46d8-bd40-cf53a3c3d5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:36.240594 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284822c-950f-46d8-bd40-cf53a3c3d5aa None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:36.241057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284822c-950f-46d8-bd40-cf53a3c3d5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:36.241638 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284822c-950f-46d8-bd40-cf53a3c3d5aa None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:36.242988 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284822c-950f-46d8-bd40-cf53a3c3d5aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:36.243520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284822c-950f-46d8-bd40-cf53a3c3d5aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:36.246182 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284822c-950f-46d8-bd40-cf53a3c3d5aa None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:36.246653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c11a1-02a1-44cc-854f-86926fe3e75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:36.248650 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c11a1-02a1-44cc-854f-86926fe3e75f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:36.249196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c11a1-02a1-44cc-854f-86926fe3e75f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:36.249726 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c11a1-02a1-44cc-854f-86926fe3e75f None admin] PATH_INFO: `/v3/users/ea7ca5941bd2450fa6784ece7e864dc7` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:36.252535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c11a1-02a1-44cc-854f-86926fe3e75f None admin] RBAC: Authorizing `identity:get_user(user_id=ea7ca5941bd2450fa6784ece7e864dc7)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:36.255201 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c11a1-02a1-44cc-854f-86926fe3e75f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:36.258519 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 186/366] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:20:35 2022] GET /identity/v3/users/ea7ca5941bd2450fa6784ece7e864dc7 => generated 293 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:36.259939 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c11a1-02a1-44cc-854f-86926fe3e75f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:36.261911 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 181/367] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:20:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:36.263408 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284822c-950f-46d8-bd40-cf53a3c3d5aa None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:36.315389 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:36.396424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c11a1-02a1-44cc-854f-86926fe3e75f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:36.412274 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:36.439609 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19b243-c4a9-44ed-b432-ca9542fad745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:36.440762 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19b243-c4a9-44ed-b432-ca9542fad745 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:36.441234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19b243-c4a9-44ed-b432-ca9542fad745 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:36.441677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19b243-c4a9-44ed-b432-ca9542fad745 None admin] PATH_INFO: `/v3/groups/6e193bbfa6174cbdb7b4484137920fb2` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:36.443594 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19b243-c4a9-44ed-b432-ca9542fad745 None admin] RBAC: Authorizing `identity:get_group(group_id=6e193bbfa6174cbdb7b4484137920fb2)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:36.445244 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19b243-c4a9-44ed-b432-ca9542fad745 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:36.447486 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 187/368] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 14 20:20:36 2022] GET /identity/v3/groups/6e193bbfa6174cbdb7b4484137920fb2 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:36.448537 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19b243-c4a9-44ed-b432-ca9542fad745 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:36.458471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284822c-950f-46d8-bd40-cf53a3c3d5aa None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:36.522191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a3abe-8b64-4d6f-b1b8-23057f5eae2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:36.523541 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a3abe-8b64-4d6f-b1b8-23057f5eae2a None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:36.524053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a3abe-8b64-4d6f-b1b8-23057f5eae2a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:36.524535 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a3abe-8b64-4d6f-b1b8-23057f5eae2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:36.548281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a3abe-8b64-4d6f-b1b8-23057f5eae2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:36.558984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-660a3abe-8b64-4d6f-b1b8-23057f5eae2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:20:36.558984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:36.558984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:36.558984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:36.558984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:36.558984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:36.558984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:36.558984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:36.558984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:36.558984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:36.558984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:36.558984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:36.558984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:36.558984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:36.558984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:36.558984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:36.559913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:36.559913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:36.559913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:36.559913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:36.559913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:36.559913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:36.559913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:36.559913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:36.559913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:36.559913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:36.559913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:36.559913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:36.559913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:36.559913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:36.559913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:36.559913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:36.561276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:36.561276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:36.561276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:36.561276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:36.561276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:36.561276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:36.561276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:36.561276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:36.561276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:36.561276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:36.561276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:20:36.561276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:36.564637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 182/369] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:20:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:36.565788 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a3abe-8b64-4d6f-b1b8-23057f5eae2a None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:36.575313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:36.583823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19b243-c4a9-44ed-b432-ca9542fad745 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:36.646150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcb365-7597-4f16-877c-ee7e6a2eab78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:36.646818 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcb365-7597-4f16-877c-ee7e6a2eab78 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:36.648377 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcb365-7597-4f16-877c-ee7e6a2eab78 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:36.648663 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcb365-7597-4f16-877c-ee7e6a2eab78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:36.649500 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdcb365-7597-4f16-877c-ee7e6a2eab78 None admin] RBAC: Adding query filter params (role.id=a7d155dd59914d22a32012302fa6d2d2, user.id=ea7ca5941bd2450fa6784ece7e864dc7, scope.system=all) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:36.649939 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdcb365-7597-4f16-877c-ee7e6a2eab78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:36.651295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdcb365-7597-4f16-877c-ee7e6a2eab78 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:36.683399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 188/370] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Jul 14 20:20:36 2022] GET /identity/v3/role_assignments?user.id=ea7ca5941bd2450fa6784ece7e864dc7&scope.system=all&role.id=a7d155dd59914d22a32012302fa6d2d2 => generated 510 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:36.684451 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcb365-7597-4f16-877c-ee7e6a2eab78 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:36.709772 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a3abe-8b64-4d6f-b1b8-23057f5eae2a None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:36.756523 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20737018-6d20-4c93-81eb-1c4006ef490e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:36.757668 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20737018-6d20-4c93-81eb-1c4006ef490e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:36.758267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20737018-6d20-4c93-81eb-1c4006ef490e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:36.758790 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20737018-6d20-4c93-81eb-1c4006ef490e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:36.763401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:36.853578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20737018-6d20-4c93-81eb-1c4006ef490e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:36.855193 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20737018-6d20-4c93-81eb-1c4006ef490e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:36.890544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcb365-7597-4f16-877c-ee7e6a2eab78 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:36.951230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd2d65-d489-47db-b92c-d3a8233af011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:36.951887 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd2d65-d489-47db-b92c-d3a8233af011 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:36.952197 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd2d65-d489-47db-b92c-d3a8233af011 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:36.952452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd2d65-d489-47db-b92c-d3a8233af011 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:36.953581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd2d65-d489-47db-b92c-d3a8233af011 None admin] RBAC: Adding query filter params (group.id=6e193bbfa6174cbdb7b4484137920fb2, role.id=b778e530d54948c7bc31f5b9231ea288, scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:36.953905 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd2d65-d489-47db-b92c-d3a8233af011 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:36.955823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd2d65-d489-47db-b92c-d3a8233af011 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:36.962724 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 189/371] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 14 20:20:36 2022] GET /identity/v3/role_assignments?group.id=6e193bbfa6174cbdb7b4484137920fb2&scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5&role.id=b778e530d54948c7bc31f5b9231ea288 => generated 260 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:36.963537 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd2d65-d489-47db-b92c-d3a8233af011 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:36.996418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:36.999242 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 183/372] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:37.000862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5909b88-7e9c-4b64-986d-0906d1ed6ad3 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:37.001258 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5909b88-7e9c-4b64-986d-0906d1ed6ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:37.001637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5909b88-7e9c-4b64-986d-0906d1ed6ad3 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:37.004063 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 184/373] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:37.006179 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5909b88-7e9c-4b64-986d-0906d1ed6ad3 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:37.055708 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:37.130411 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd2d65-d489-47db-b92c-d3a8233af011 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:37.138493 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5909b88-7e9c-4b64-986d-0906d1ed6ad3 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:37.176622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985a89b-21ca-454e-a0b3-e51c902bee87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:37.177467 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985a89b-21ca-454e-a0b3-e51c902bee87 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:37.177764 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985a89b-21ca-454e-a0b3-e51c902bee87 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:37.178088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985a89b-21ca-454e-a0b3-e51c902bee87 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:37.179438 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985a89b-21ca-454e-a0b3-e51c902bee87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:37.179438 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985a89b-21ca-454e-a0b3-e51c902bee87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:37.180842 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985a89b-21ca-454e-a0b3-e51c902bee87 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:37.181219 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055eb2a4-985b-4766-93fd-9d5780371c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:37.182362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055eb2a4-985b-4766-93fd-9d5780371c4c None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:37.182858 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055eb2a4-985b-4766-93fd-9d5780371c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:37.183387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055eb2a4-985b-4766-93fd-9d5780371c4c None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:37.184895 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055eb2a4-985b-4766-93fd-9d5780371c4c None admin] RBAC: Authorizing `identity:get_project(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:37.186646 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055eb2a4-985b-4766-93fd-9d5780371c4c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:37.189054 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 185/374] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:37 2022] GET /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:37.194994 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 190/375] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:37.195915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985a89b-21ca-454e-a0b3-e51c902bee87 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:37.196166 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ebae8-95f6-470a-af72-ab67a9528b95 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:37.196674 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ebae8-95f6-470a-af72-ab67a9528b95 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:37.197158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ebae8-95f6-470a-af72-ab67a9528b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:37.220099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-bd9ebae8-95f6-470a-af72-ab67a9528b95 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:37.221250 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-bd9ebae8-95f6-470a-af72-ab67a9528b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:37.301723 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:37.312972 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 186/376] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:37.318998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ebae8-95f6-470a-af72-ab67a9528b95 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:37.355599 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:37.359240 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985a89b-21ca-454e-a0b3-e51c902bee87 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:37.410174 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6680d-aa12-4d7c-8ce8-e685e8a45515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:37.411401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6680d-aa12-4d7c-8ce8-e685e8a45515 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:37.411758 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6680d-aa12-4d7c-8ce8-e685e8a45515 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:37.412477 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6680d-aa12-4d7c-8ce8-e685e8a45515 None admin] PATH_INFO: `/v3/groups/6e193bbfa6174cbdb7b4484137920fb2` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:37.414076 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6680d-aa12-4d7c-8ce8-e685e8a45515 None admin] RBAC: Authorizing `identity:get_group(group_id=6e193bbfa6174cbdb7b4484137920fb2)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:37.415587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6680d-aa12-4d7c-8ce8-e685e8a45515 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:37.418025 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 191/377] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 14 20:20:37 2022] GET /identity/v3/groups/6e193bbfa6174cbdb7b4484137920fb2 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:37.419018 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6680d-aa12-4d7c-8ce8-e685e8a45515 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:37.464872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ebae8-95f6-470a-af72-ab67a9528b95 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:37.508292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d15723-7ba0-420e-b48e-9a5206949191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:37.509008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d15723-7ba0-420e-b48e-9a5206949191 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:37.509289 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d15723-7ba0-420e-b48e-9a5206949191 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:37.509564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d15723-7ba0-420e-b48e-9a5206949191 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:37.563993 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6680d-aa12-4d7c-8ce8-e685e8a45515 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:37.596462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d15723-7ba0-420e-b48e-9a5206949191 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:37.598108 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d15723-7ba0-420e-b48e-9a5206949191 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:37.608031 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430e7e9-03a8-4d58-ac18-c6b03a494ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:37.608841 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430e7e9-03a8-4d58-ac18-c6b03a494ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:37.609132 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430e7e9-03a8-4d58-ac18-c6b03a494ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:37.609408 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430e7e9-03a8-4d58-ac18-c6b03a494ef8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:37.624974 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430e7e9-03a8-4d58-ac18-c6b03a494ef8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:37.633776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-f430e7e9-03a8-4d58-ac18-c6b03a494ef8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:37.633776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:37.633776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:37.633776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:37.633776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:37.633776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:37.633776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:37.633776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:37.633776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:37.633776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:37.633776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:37.633776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:37.633776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:37.633776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:37.633776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:37.633776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:37.634713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:37.634713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:37.634713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:37.634713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:37.634713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:37.634713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:37.634713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:37.634713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:37.634713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:37.634713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:37.634713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:37.634713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:37.634713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:37.634713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:37.634713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:37.634713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:37.635592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:37.635592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:37.635592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:37.635592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:37.635592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:37.635592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:37.635592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:37.635592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:37.635592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:37.635592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:37.635592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:37.635592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:37.636230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 192/378] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:37.636784 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430e7e9-03a8-4d58-ac18-c6b03a494ef8 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:37.650243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:37.652375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:37.740494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 187/379] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:37.741470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d15723-7ba0-420e-b48e-9a5206949191 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:37.782868 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430e7e9-03a8-4d58-ac18-c6b03a494ef8 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:37.829611 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17601c36-3dff-4370-a4c0-f93bc20dd705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:37.830282 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17601c36-3dff-4370-a4c0-f93bc20dd705 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:37.830569 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17601c36-3dff-4370-a4c0-f93bc20dd705 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:37.830846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17601c36-3dff-4370-a4c0-f93bc20dd705 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:37.831726 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17601c36-3dff-4370-a4c0-f93bc20dd705 None admin] RBAC: Adding query filter params (group.id=6e193bbfa6174cbdb7b4484137920fb2, role.id=6b2eb3e4cefd4a2db4f18e04a158600a, scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:37.832043 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17601c36-3dff-4370-a4c0-f93bc20dd705 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:37.833520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17601c36-3dff-4370-a4c0-f93bc20dd705 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:37.840130 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 193/380] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 14 20:20:37 2022] GET /identity/v3/role_assignments?group.id=6e193bbfa6174cbdb7b4484137920fb2&scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5&role.id=6b2eb3e4cefd4a2db4f18e04a158600a => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:37.847864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17601c36-3dff-4370-a4c0-f93bc20dd705 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:37.865083 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d15723-7ba0-420e-b48e-9a5206949191 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:37.906616 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfb762-96ae-430b-aced-998bbded9ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:37.907325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfb762-96ae-430b-aced-998bbded9ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:37.907667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfb762-96ae-430b-aced-998bbded9ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:37.907942 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfb762-96ae-430b-aced-998bbded9ec3 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:37.908775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfb762-96ae-430b-aced-998bbded9ec3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:37.909089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfb762-96ae-430b-aced-998bbded9ec3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:37.910564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfb762-96ae-430b-aced-998bbded9ec3 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:37.912237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:37.923655 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 188/381] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:37.931626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfb762-96ae-430b-aced-998bbded9ec3 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:37.977151 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:38.046318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17601c36-3dff-4370-a4c0-f93bc20dd705 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:38.101393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea9a87-5ee5-4e83-8c2e-501e7f1654e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:38.102289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea9a87-5ee5-4e83-8c2e-501e7f1654e1 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:38.102576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea9a87-5ee5-4e83-8c2e-501e7f1654e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:38.102852 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea9a87-5ee5-4e83-8c2e-501e7f1654e1 None admin] PATH_INFO: `/v3/projects/a9054cd3a81847fdaa8f38a717dc0938` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:38.105220 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea9a87-5ee5-4e83-8c2e-501e7f1654e1 None admin] RBAC: Authorizing `identity:get_project(project_id=a9054cd3a81847fdaa8f38a717dc0938)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:38.106703 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea9a87-5ee5-4e83-8c2e-501e7f1654e1 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:38.109085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 194/382] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:37 2022] GET /identity/v3/projects/a9054cd3a81847fdaa8f38a717dc0938 => generated 344 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:38.116931 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea9a87-5ee5-4e83-8c2e-501e7f1654e1 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:38.135027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfb762-96ae-430b-aced-998bbded9ec3 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:38.180684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b950b2-647a-4a74-92cd-80ec1bc9ce2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:38.181535 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b950b2-647a-4a74-92cd-80ec1bc9ce2e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:38.181820 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b950b2-647a-4a74-92cd-80ec1bc9ce2e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:38.182099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b950b2-647a-4a74-92cd-80ec1bc9ce2e None admin] PATH_INFO: `/v3/groups/6e193bbfa6174cbdb7b4484137920fb2` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:38.183456 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b950b2-647a-4a74-92cd-80ec1bc9ce2e None admin] RBAC: Authorizing `identity:get_group(group_id=6e193bbfa6174cbdb7b4484137920fb2)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:38.186544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b950b2-647a-4a74-92cd-80ec1bc9ce2e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:38.189011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 189/383] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 14 20:20:37 2022] GET /identity/v3/groups/6e193bbfa6174cbdb7b4484137920fb2 => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:38.198075 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b950b2-647a-4a74-92cd-80ec1bc9ce2e None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:38.305900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea9a87-5ee5-4e83-8c2e-501e7f1654e1 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:38.332342 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:38.377615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c8b26-2b66-4244-98fb-5f140f4c282b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:38.378768 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c8b26-2b66-4244-98fb-5f140f4c282b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:38.379222 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c8b26-2b66-4244-98fb-5f140f4c282b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:38.379641 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c8b26-2b66-4244-98fb-5f140f4c282b None admin] PATH_INFO: `/v3/groups/e90f520365d74e5ca16ea41ee94bb18a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:38.388762 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c8b26-2b66-4244-98fb-5f140f4c282b None admin] RBAC: Authorizing `identity:get_group(group_id=e90f520365d74e5ca16ea41ee94bb18a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:38.390481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c8b26-2b66-4244-98fb-5f140f4c282b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:38.392427 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 195/384] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 14 20:20:38 2022] GET /identity/v3/groups/e90f520365d74e5ca16ea41ee94bb18a => generated 229 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:38.394204 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47b0c6-2114-4f49-9f7d-ca72f5e8768f None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:38.394713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47b0c6-2114-4f49-9f7d-ca72f5e8768f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:38.395222 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47b0c6-2114-4f49-9f7d-ca72f5e8768f None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:38.397041 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 196/385] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:38.403442 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47b0c6-2114-4f49-9f7d-ca72f5e8768f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:38.405959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b950b2-647a-4a74-92cd-80ec1bc9ce2e None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:38.407798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:38.449678 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957d57e-013b-494f-bc50-2eb6b5e1d316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:38.450561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957d57e-013b-494f-bc50-2eb6b5e1d316 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:38.450887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957d57e-013b-494f-bc50-2eb6b5e1d316 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:38.451183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957d57e-013b-494f-bc50-2eb6b5e1d316 None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:38.452715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957d57e-013b-494f-bc50-2eb6b5e1d316 None admin] RBAC: Authorizing `identity:get_project(project_id=930b378be44b4bb7a2009c72a424f1d6)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:38.454166 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957d57e-013b-494f-bc50-2eb6b5e1d316 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:38.457106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 190/386] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:38 2022] GET /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:38.458811 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2663f0-5c33-4426-a73f-a0e8f410e22d None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:38.459121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2663f0-5c33-4426-a73f-a0e8f410e22d None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:38.459399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2663f0-5c33-4426-a73f-a0e8f410e22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:38.478476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-0a2663f0-5c33-4426-a73f-a0e8f410e22d None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:38.481136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-0a2663f0-5c33-4426-a73f-a0e8f410e22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:38.553580 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47b0c6-2114-4f49-9f7d-ca72f5e8768f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:38.569345 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 191/387] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:38.570127 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2663f0-5c33-4426-a73f-a0e8f410e22d None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:38.613389 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5d03b-6f17-4dea-a335-c53fee1c6eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:38.614927 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5d03b-6f17-4dea-a335-c53fee1c6eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:38.615234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5d03b-6f17-4dea-a335-c53fee1c6eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:38.615507 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5d03b-6f17-4dea-a335-c53fee1c6eb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:38.619161 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5d03b-6f17-4dea-a335-c53fee1c6eb5 None admin] RBAC: Adding query filter params (group.id=e90f520365d74e5ca16ea41ee94bb18a, role.id=5ca112885467422684e134cb0f94865a, scope.project.id=a9054cd3a81847fdaa8f38a717dc0938) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:38.619573 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5d03b-6f17-4dea-a335-c53fee1c6eb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:38.621111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5d03b-6f17-4dea-a335-c53fee1c6eb5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:38.628423 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 197/388] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 14 20:20:38 2022] GET /identity/v3/role_assignments?group.id=e90f520365d74e5ca16ea41ee94bb18a&scope.project.id=a9054cd3a81847fdaa8f38a717dc0938&role.id=5ca112885467422684e134cb0f94865a => generated 260 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:38.629288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5d03b-6f17-4dea-a335-c53fee1c6eb5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:38.667709 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:38.715884 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2663f0-5c33-4426-a73f-a0e8f410e22d None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:38.756844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5935ce0-dc23-47cc-90de-318ca57260c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:38.757768 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5935ce0-dc23-47cc-90de-318ca57260c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:38.758050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5935ce0-dc23-47cc-90de-318ca57260c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:38.758329 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5935ce0-dc23-47cc-90de-318ca57260c6 None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6/groups/6e193bbfa6174cbdb7b4484137920fb2/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:38.760852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5935ce0-dc23-47cc-90de-318ca57260c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=930b378be44b4bb7a2009c72a424f1d6, group_id=6e193bbfa6174cbdb7b4484137920fb2, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:38.762502 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5935ce0-dc23-47cc-90de-318ca57260c6 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:38.787753 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 192/389] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jul 14 20:20:38 2022] PUT /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6/groups/6e193bbfa6174cbdb7b4484137920fb2/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:20:38.819484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:38.842606 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5d03b-6f17-4dea-a335-c53fee1c6eb5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:38.887101 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d670c33-8b51-48fb-8481-38581a3b5211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:38.887924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d670c33-8b51-48fb-8481-38581a3b5211 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:38.888206 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d670c33-8b51-48fb-8481-38581a3b5211 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:38.888481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d670c33-8b51-48fb-8481-38581a3b5211 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:38.901143 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d670c33-8b51-48fb-8481-38581a3b5211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:38.909484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-7d670c33-8b51-48fb-8481-38581a3b5211 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:38.909484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:38.909484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:38.909484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:38.909484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:38.909484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:38.909484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:38.909484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:38.909484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:38.909484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:38.909484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:38.909484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:38.909484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:38.909484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:38.909484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:38.909484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:38.910430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:38.910430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:38.910430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:38.910430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:38.910430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:38.910430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:38.910430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:38.910430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:38.910430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:38.910430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:38.910430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:38.910430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:38.910430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:38.910430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:38.910430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:38.910430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:38.912078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:38.912078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:38.912078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:38.912078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:38.912078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:38.912078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:38.912078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:38.912078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:38.912078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:38.912078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:38.912078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:38.912078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:38.912078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 198/390] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:20:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:38.917036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5935ce0-dc23-47cc-90de-318ca57260c6 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:39.077063 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5935ce0-dc23-47cc-90de-318ca57260c6 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:39.105875 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:39.123495 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb83681-7328-41f4-b2fd-f589198480cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:39.124527 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb83681-7328-41f4-b2fd-f589198480cf None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:39.125080 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb83681-7328-41f4-b2fd-f589198480cf None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:39.125549 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb83681-7328-41f4-b2fd-f589198480cf None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:39.126568 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb83681-7328-41f4-b2fd-f589198480cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:39.127183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb83681-7328-41f4-b2fd-f589198480cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:39.128773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb83681-7328-41f4-b2fd-f589198480cf None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:39.139911 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 193/391] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:20:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:39.148053 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d670c33-8b51-48fb-8481-38581a3b5211 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:39.315330 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d670c33-8b51-48fb-8481-38581a3b5211 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:39.347389 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:39.361582 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2c615-5f04-4513-a272-f725bf86ff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:39.362474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2c615-5f04-4513-a272-f725bf86ff3d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:39.362755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2c615-5f04-4513-a272-f725bf86ff3d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:39.363052 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2c615-5f04-4513-a272-f725bf86ff3d None admin] PATH_INFO: `/v3/groups/6e193bbfa6174cbdb7b4484137920fb2` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:39.364264 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2c615-5f04-4513-a272-f725bf86ff3d None admin] RBAC: Authorizing `identity:get_group(group_id=6e193bbfa6174cbdb7b4484137920fb2)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:39.365736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2c615-5f04-4513-a272-f725bf86ff3d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:39.367953 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 199/392] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 14 20:20:39 2022] GET /identity/v3/groups/6e193bbfa6174cbdb7b4484137920fb2 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:39.373829 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb83681-7328-41f4-b2fd-f589198480cf None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:39.518243 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb83681-7328-41f4-b2fd-f589198480cf None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:39.559610 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d1285-b648-426e-afea-d29e20395b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:39.560492 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d1285-b648-426e-afea-d29e20395b59 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:39.560819 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d1285-b648-426e-afea-d29e20395b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:39.561098 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d1285-b648-426e-afea-d29e20395b59 None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:39.563112 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d1285-b648-426e-afea-d29e20395b59 None admin] RBAC: Authorizing `identity:get_project(project_id=930b378be44b4bb7a2009c72a424f1d6)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:39.564590 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d1285-b648-426e-afea-d29e20395b59 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:39.566694 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 194/393] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:39 2022] GET /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:39.572358 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2c615-5f04-4513-a272-f725bf86ff3d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:39.695853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6f867-c034-4cb3-a478-2ee8c166b928 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:39.696192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6f867-c034-4cb3-a478-2ee8c166b928 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:39.696420 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6f867-c034-4cb3-a478-2ee8c166b928 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:39.697832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 195/394] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:20:39.713538 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22889a8-d648-491f-93f4-3e6f3b16bfcb None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:39.713823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22889a8-d648-491f-93f4-3e6f3b16bfcb None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:39.714101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22889a8-d648-491f-93f4-3e6f3b16bfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:39.739940 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-b22889a8-d648-491f-93f4-3e6f3b16bfcb None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:39.741170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b22889a8-d648-491f-93f4-3e6f3b16bfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:39.750735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2c615-5f04-4513-a272-f725bf86ff3d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:39.822784 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b337ea-0708-4b79-9257-598810cbaf61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:39.823868 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b337ea-0708-4b79-9257-598810cbaf61 None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:39.824213 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b337ea-0708-4b79-9257-598810cbaf61 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:39.824496 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b337ea-0708-4b79-9257-598810cbaf61 None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6/groups/6e193bbfa6174cbdb7b4484137920fb2/roles/6b2eb3e4cefd4a2db4f18e04a158600a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:39.831630 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:39.834091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b337ea-0708-4b79-9257-598810cbaf61 None admin] RBAC: Authorizing `identity:create_grant(project_id=930b378be44b4bb7a2009c72a424f1d6, group_id=6e193bbfa6174cbdb7b4484137920fb2, role_id=6b2eb3e4cefd4a2db4f18e04a158600a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:39.836336 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b337ea-0708-4b79-9257-598810cbaf61 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:39.854633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 196/395] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:20:39.861365 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 200/396] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jul 14 20:20:39 2022] PUT /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6/groups/6e193bbfa6174cbdb7b4484137920fb2/roles/6b2eb3e4cefd4a2db4f18e04a158600a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:20:39.898802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22889a8-d648-491f-93f4-3e6f3b16bfcb None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:40.049835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22889a8-d648-491f-93f4-3e6f3b16bfcb None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:40.089100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0f6aa-2991-4405-9ed9-efe2289a92f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:40.089926 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0f6aa-2991-4405-9ed9-efe2289a92f5 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:40.090210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0f6aa-2991-4405-9ed9-efe2289a92f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:40.090488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0f6aa-2991-4405-9ed9-efe2289a92f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:40.107580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0f6aa-2991-4405-9ed9-efe2289a92f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:40.115418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-35b0f6aa-2991-4405-9ed9-efe2289a92f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:40.115418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:40.115418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:40.115418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:40.115418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:40.115418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:40.115418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:40.115418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:40.115418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:40.115418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:40.115418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:40.115418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:40.115418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:40.115418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:40.115418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:40.115418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:40.116337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:40.116337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:40.116337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:40.116337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:40.116337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:40.116337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:40.116337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:40.116337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:40.116337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:40.116337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:40.116337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:40.116337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:40.116337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:40.116337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:40.116337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:40.116337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:40.117148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:40.117148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:40.117148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:40.117148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:40.117148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:40.117148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:40.117148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:40.117148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:40.117148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:40.117148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:40.117148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:40.117148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:40.117148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 197/397] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:40.122824 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b337ea-0708-4b79-9257-598810cbaf61 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:40.193297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:40.252866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b337ea-0708-4b79-9257-598810cbaf61 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:40.293951 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b32491-eae0-4b57-b2b4-535ca2a99ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:40.294659 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b32491-eae0-4b57-b2b4-535ca2a99ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:40.294990 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b32491-eae0-4b57-b2b4-535ca2a99ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:40.295277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b32491-eae0-4b57-b2b4-535ca2a99ba7 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:40.296051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b32491-eae0-4b57-b2b4-535ca2a99ba7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:40.296360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b32491-eae0-4b57-b2b4-535ca2a99ba7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:40.297775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b32491-eae0-4b57-b2b4-535ca2a99ba7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:40.308647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 201/398] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:40.313029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0f6aa-2991-4405-9ed9-efe2289a92f5 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:40.395103 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:40.468019 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0f6aa-2991-4405-9ed9-efe2289a92f5 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:40.512771 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bf46e-e27c-444c-97ed-8e9ef8181546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:40.513641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bf46e-e27c-444c-97ed-8e9ef8181546 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:40.514473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bf46e-e27c-444c-97ed-8e9ef8181546 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:40.514748 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bf46e-e27c-444c-97ed-8e9ef8181546 None admin] PATH_INFO: `/v3/groups/6e193bbfa6174cbdb7b4484137920fb2` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:40.515994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bf46e-e27c-444c-97ed-8e9ef8181546 None admin] RBAC: Authorizing `identity:get_group(group_id=6e193bbfa6174cbdb7b4484137920fb2)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:40.517454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bf46e-e27c-444c-97ed-8e9ef8181546 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:40.519420 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 198/399] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 14 20:20:40 2022] GET /identity/v3/groups/6e193bbfa6174cbdb7b4484137920fb2 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:40.525032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b32491-eae0-4b57-b2b4-535ca2a99ba7 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:40.768296 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b32491-eae0-4b57-b2b4-535ca2a99ba7 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:40.793621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:40.818370 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da918e1-1bf8-4083-8067-d98d6d885c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:40.819302 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da918e1-1bf8-4083-8067-d98d6d885c91 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:40.819584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da918e1-1bf8-4083-8067-d98d6d885c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:40.819861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da918e1-1bf8-4083-8067-d98d6d885c91 None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:40.827424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da918e1-1bf8-4083-8067-d98d6d885c91 None admin] RBAC: Authorizing `identity:get_project(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:40.827607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b5acb-0710-4ba3-8ee0-9a25444d7862 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:40.827607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b5acb-0710-4ba3-8ee0-9a25444d7862 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:40.827607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b5acb-0710-4ba3-8ee0-9a25444d7862 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:40.827803 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da918e1-1bf8-4083-8067-d98d6d885c91 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:40.827885 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 199/400] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:40.828654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 202/401] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:40 2022] GET /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5 => generated 302 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:40.834073 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da918e1-1bf8-4083-8067-d98d6d885c91 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:40.834432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e0e97-fec4-4a07-8c5d-5337c1ddae42 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:40.834729 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e0e97-fec4-4a07-8c5d-5337c1ddae42 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:40.835040 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e0e97-fec4-4a07-8c5d-5337c1ddae42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:40.852526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-7c1e0e97-fec4-4a07-8c5d-5337c1ddae42 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:40.853373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-7c1e0e97-fec4-4a07-8c5d-5337c1ddae42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:40.952988 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 200/402] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:40.984573 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da918e1-1bf8-4083-8067-d98d6d885c91 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:41.012099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e0e97-fec4-4a07-8c5d-5337c1ddae42 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:41.063363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:41.070676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a2f96-2c1b-4ebb-b785-ce5f406fb1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:41.071628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a2f96-2c1b-4ebb-b785-ce5f406fb1cd None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:41.071916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a2f96-2c1b-4ebb-b785-ce5f406fb1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:41.072195 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a2f96-2c1b-4ebb-b785-ce5f406fb1cd None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5/groups/6e193bbfa6174cbdb7b4484137920fb2/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:41.075753 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a2f96-2c1b-4ebb-b785-ce5f406fb1cd None admin] RBAC: Authorizing `identity:create_grant(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5, group_id=6e193bbfa6174cbdb7b4484137920fb2, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:41.077277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a2f96-2c1b-4ebb-b785-ce5f406fb1cd None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:41.105883 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 203/403] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jul 14 20:20:40 2022] PUT /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5/groups/6e193bbfa6174cbdb7b4484137920fb2/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:20:41.159961 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e0e97-fec4-4a07-8c5d-5337c1ddae42 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:41.199237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36359737-77f0-4ecd-b2d2-4f35d4935cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:41.200076 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36359737-77f0-4ecd-b2d2-4f35d4935cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:41.200363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36359737-77f0-4ecd-b2d2-4f35d4935cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:41.200644 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36359737-77f0-4ecd-b2d2-4f35d4935cd7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:41.212810 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36359737-77f0-4ecd-b2d2-4f35d4935cd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:41.226161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-36359737-77f0-4ecd-b2d2-4f35d4935cd7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:41.226161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:41.226161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:41.226161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:41.226161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:41.226161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:41.226161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:41.226161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:41.226161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:41.226161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:41.226161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:41.226161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:41.226161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:41.226161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:41.226161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:41.226161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:41.227135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:41.227135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:41.227135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:41.227135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:41.227135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:41.227135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:41.227135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:41.227135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:41.227135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:41.227135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:41.227135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:41.227135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:41.227135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:41.227135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:41.227135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:41.227135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:41.228014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:41.228014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:41.228014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:41.228014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:41.228014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:41.228014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:41.228014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:41.228014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:41.228014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:41.228014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:41.228014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:41.228014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:41.228014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 201/404] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:20:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:41.234407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a2f96-2c1b-4ebb-b785-ce5f406fb1cd None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:41.302081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:41.382841 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a2f96-2c1b-4ebb-b785-ce5f406fb1cd None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:41.384520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1a385-c0ab-4dcb-8eb0-9422599d208b None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:41.384986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1a385-c0ab-4dcb-8eb0-9422599d208b None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:41.385075 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1a385-c0ab-4dcb-8eb0-9422599d208b None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:41.386659 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 202/405] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:41.396470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4c36b-994b-4ad2-a7a4-e4b59a5f4a55 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:41.396746 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4c36b-994b-4ad2-a7a4-e4b59a5f4a55 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:41.397032 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4c36b-994b-4ad2-a7a4-e4b59a5f4a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:41.418106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-64a4c36b-994b-4ad2-a7a4-e4b59a5f4a55 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:41.419390 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-64a4c36b-994b-4ad2-a7a4-e4b59a5f4a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:41.425047 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d235868-3dad-4d6f-b1fb-0195a2dbf3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:41.425979 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d235868-3dad-4d6f-b1fb-0195a2dbf3bd None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:41.426412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d235868-3dad-4d6f-b1fb-0195a2dbf3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:41.426827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d235868-3dad-4d6f-b1fb-0195a2dbf3bd None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:41.427848 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d235868-3dad-4d6f-b1fb-0195a2dbf3bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:41.428297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d235868-3dad-4d6f-b1fb-0195a2dbf3bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:41.429950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d235868-3dad-4d6f-b1fb-0195a2dbf3bd None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:41.446059 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 204/406] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:20:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:41.448053 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d235868-3dad-4d6f-b1fb-0195a2dbf3bd None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:41.521726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 203/407] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:41.523211 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6602b22-25ff-40b1-a9c0-b1a9878418e1 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:41.523500 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6602b22-25ff-40b1-a9c0-b1a9878418e1 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:41.523775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6602b22-25ff-40b1-a9c0-b1a9878418e1 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:41.525195 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 204/408] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:41.537481 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9331d1-dd25-4f28-b1a8-457a7e0cbda7 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:41.537783 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9331d1-dd25-4f28-b1a8-457a7e0cbda7 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:41.538040 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9331d1-dd25-4f28-b1a8-457a7e0cbda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:41.565462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-5f9331d1-dd25-4f28-b1a8-457a7e0cbda7 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:41.566677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-5f9331d1-dd25-4f28-b1a8-457a7e0cbda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:41.588878 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d235868-3dad-4d6f-b1fb-0195a2dbf3bd None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:41.593099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:41.601045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:41.649850 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a2fa6-b732-40d3-81cf-304926ae0c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:41.650634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a2fa6-b732-40d3-81cf-304926ae0c6c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:41.650917 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a2fa6-b732-40d3-81cf-304926ae0c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:41.651225 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a2fa6-b732-40d3-81cf-304926ae0c6c None admin] PATH_INFO: `/v3/groups/6e193bbfa6174cbdb7b4484137920fb2` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:41.652631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a2fa6-b732-40d3-81cf-304926ae0c6c None admin] RBAC: Authorizing `identity:get_group(group_id=6e193bbfa6174cbdb7b4484137920fb2)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:41.656026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 205/409] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:41.656244 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a2fa6-b732-40d3-81cf-304926ae0c6c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:41.658058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9331d1-dd25-4f28-b1a8-457a7e0cbda7 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:41.669056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 205/410] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 14 20:20:41 2022] GET /identity/v3/groups/6e193bbfa6174cbdb7b4484137920fb2 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:41.671910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a2fa6-b732-40d3-81cf-304926ae0c6c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:41.814786 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9331d1-dd25-4f28-b1a8-457a7e0cbda7 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:41.830968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a2fa6-b732-40d3-81cf-304926ae0c6c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:41.869786 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064cc43-6267-4704-92da-a2ef471482ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:41.871077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064cc43-6267-4704-92da-a2ef471482ea None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:41.871753 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064cc43-6267-4704-92da-a2ef471482ea None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:41.872234 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064cc43-6267-4704-92da-a2ef471482ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:41.877535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab283c35-8495-42e7-875d-ee1bb9ac0956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:41.878409 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab283c35-8495-42e7-875d-ee1bb9ac0956 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:41.878689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab283c35-8495-42e7-875d-ee1bb9ac0956 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:41.878995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab283c35-8495-42e7-875d-ee1bb9ac0956 None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:41.880559 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab283c35-8495-42e7-875d-ee1bb9ac0956 None admin] RBAC: Authorizing `identity:get_project(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:41.882025 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab283c35-8495-42e7-875d-ee1bb9ac0956 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:41.885074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 206/411] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:41 2022] GET /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:41.886299 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab283c35-8495-42e7-875d-ee1bb9ac0956 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:41.887258 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064cc43-6267-4704-92da-a2ef471482ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:41.896606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-4064cc43-6267-4704-92da-a2ef471482ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:41.896606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:41.896606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:41.896606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:41.896606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:41.896606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:41.896606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:41.896606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:41.896606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:41.896606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:41.896606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:41.896606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:41.896606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:41.896606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:41.896606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:41.896606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:41.897550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:41.897550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:41.897550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:41.897550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:41.897550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:41.897550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:41.897550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:41.897550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:41.897550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:41.897550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:41.897550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:41.897550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:41.897550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:41.897550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:41.897550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:41.897550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:41.898366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:41.898366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:41.898366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:41.898366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:41.898366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:41.898366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:41.898366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:41.898366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:41.898366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:41.898366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:41.898366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:41.898366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:41.906346 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 206/412] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:41.907630 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064cc43-6267-4704-92da-a2ef471482ea None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:41.941852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:41.944248 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:42.020702 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab283c35-8495-42e7-875d-ee1bb9ac0956 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:42.051563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064cc43-6267-4704-92da-a2ef471482ea None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:42.070349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04751bd-b252-402a-80da-8a8edc454442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:42.071572 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04751bd-b252-402a-80da-8a8edc454442 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:42.072182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04751bd-b252-402a-80da-8a8edc454442 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:42.072663 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04751bd-b252-402a-80da-8a8edc454442 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:42.084940 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04751bd-b252-402a-80da-8a8edc454442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:42.094615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-d04751bd-b252-402a-80da-8a8edc454442 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:20:42.094615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:42.094615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:42.094615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:42.094615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:42.094615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:42.094615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:42.094615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:42.094615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:42.094615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:42.094615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:42.094615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:42.094615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:42.094615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:42.094615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:42.094615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:42.095616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:42.095616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:42.095616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:42.095616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:42.095616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:42.095616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:42.095616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:42.095616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:42.095616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:42.095616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:42.095616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:42.095616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:42.095616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:42.095616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:42.095616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:42.095616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:42.096412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:42.096412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:42.096412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:42.096412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:42.096412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:42.096412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:42.096412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:42.096412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:42.096412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:42.096412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:42.096412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:20:42.096412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:42.103804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 207/413] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:20:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:42.103804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04751bd-b252-402a-80da-8a8edc454442 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:42.104674 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8c52c-94fb-4792-bb25-f503e417e13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:42.105580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8c52c-94fb-4792-bb25-f503e417e13d None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:42.105859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8c52c-94fb-4792-bb25-f503e417e13d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:42.106133 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8c52c-94fb-4792-bb25-f503e417e13d None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5/groups/6e193bbfa6174cbdb7b4484137920fb2/roles/6b2eb3e4cefd4a2db4f18e04a158600a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:42.112246 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c8c52c-94fb-4792-bb25-f503e417e13d None admin] RBAC: Authorizing `identity:create_grant(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5, group_id=6e193bbfa6174cbdb7b4484137920fb2, role_id=6b2eb3e4cefd4a2db4f18e04a158600a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:42.114074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c8c52c-94fb-4792-bb25-f503e417e13d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:42.131670 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 207/414] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jul 14 20:20:41 2022] PUT /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5/groups/6e193bbfa6174cbdb7b4484137920fb2/roles/6b2eb3e4cefd4a2db4f18e04a158600a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:20:42.132996 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8c52c-94fb-4792-bb25-f503e417e13d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:42.243581 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:42.287200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8c52c-94fb-4792-bb25-f503e417e13d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:42.334778 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04751bd-b252-402a-80da-8a8edc454442 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:42.343554 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:42.347882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4a63f-0607-4e2f-bb96-802736dd13ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:42.348892 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4a63f-0607-4e2f-bb96-802736dd13ba None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:42.349388 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4a63f-0607-4e2f-bb96-802736dd13ba None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:42.349885 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4a63f-0607-4e2f-bb96-802736dd13ba None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:42.350981 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4a63f-0607-4e2f-bb96-802736dd13ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:42.351493 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4a63f-0607-4e2f-bb96-802736dd13ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:42.353450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4a63f-0607-4e2f-bb96-802736dd13ba None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:42.366022 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 208/415] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:42.371952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4a63f-0607-4e2f-bb96-802736dd13ba None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:42.379612 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a3b88-2f54-4a34-83e9-2c0f65e074be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:42.380370 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a3b88-2f54-4a34-83e9-2c0f65e074be None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:42.380649 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a3b88-2f54-4a34-83e9-2c0f65e074be None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:42.380924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a3b88-2f54-4a34-83e9-2c0f65e074be None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:42.381686 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a3b88-2f54-4a34-83e9-2c0f65e074be None admin] RBAC: Adding query filter params (name=member) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:42.381995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a3b88-2f54-4a34-83e9-2c0f65e074be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:42.383539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a3b88-2f54-4a34-83e9-2c0f65e074be None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:42.395660 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 208/416] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:42.408026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a3b88-2f54-4a34-83e9-2c0f65e074be None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:42.501948 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4a63f-0607-4e2f-bb96-802736dd13ba None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:42.544730 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db461d8-d5d0-47d0-b4ac-5e060b913152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:42.546688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db461d8-d5d0-47d0-b4ac-5e060b913152 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:42.547209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db461d8-d5d0-47d0-b4ac-5e060b913152 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:42.547676 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db461d8-d5d0-47d0-b4ac-5e060b913152 None admin] PATH_INFO: `/v3/groups/e90f520365d74e5ca16ea41ee94bb18a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:42.549074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db461d8-d5d0-47d0-b4ac-5e060b913152 None admin] RBAC: Authorizing `identity:get_group(group_id=e90f520365d74e5ca16ea41ee94bb18a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:42.550694 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db461d8-d5d0-47d0-b4ac-5e060b913152 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:42.553474 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 209/417] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 14 20:20:42 2022] GET /identity/v3/groups/e90f520365d74e5ca16ea41ee94bb18a => generated 229 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:42.565465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db461d8-d5d0-47d0-b4ac-5e060b913152 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:42.590830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:42.651044 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a3b88-2f54-4a34-83e9-2c0f65e074be None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:42.693815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9cd49-3528-4292-97a8-34fe3067fab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:42.694612 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9cd49-3528-4292-97a8-34fe3067fab4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:42.695058 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9cd49-3528-4292-97a8-34fe3067fab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:42.695326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9cd49-3528-4292-97a8-34fe3067fab4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:42.697588 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db461d8-d5d0-47d0-b4ac-5e060b913152 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:42.731532 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:42.745850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d1f1c-8b6b-4b12-8414-528fb643fa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:42.747540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d1f1c-8b6b-4b12-8414-528fb643fa4b None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:42.748995 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d1f1c-8b6b-4b12-8414-528fb643fa4b None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:42.749603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d1f1c-8b6b-4b12-8414-528fb643fa4b None admin] PATH_INFO: `/v3/projects/a9054cd3a81847fdaa8f38a717dc0938` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:42.751120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d1f1c-8b6b-4b12-8414-528fb643fa4b None admin] RBAC: Authorizing `identity:get_project(project_id=a9054cd3a81847fdaa8f38a717dc0938)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:42.753660 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d1f1c-8b6b-4b12-8414-528fb643fa4b None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:42.756079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 210/418] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:42 2022] GET /identity/v3/projects/a9054cd3a81847fdaa8f38a717dc0938 => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:42.757565 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a773b84-7b1d-4c6e-8b39-f71f8b4d5b5b None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:42.757994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a773b84-7b1d-4c6e-8b39-f71f8b4d5b5b None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:42.758402 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a773b84-7b1d-4c6e-8b39-f71f8b4d5b5b None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:42.759798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 211/419] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:42.766035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c15ed9-5d43-44c8-9e20-9983e9b26b64 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:42.766324 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c15ed9-5d43-44c8-9e20-9983e9b26b64 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:42.766593 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c15ed9-5d43-44c8-9e20-9983e9b26b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:42.785873 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9cd49-3528-4292-97a8-34fe3067fab4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:42.787728 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9cd49-3528-4292-97a8-34fe3067fab4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:42.788826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-20c15ed9-5d43-44c8-9e20-9983e9b26b64 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:42.792768 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-20c15ed9-5d43-44c8-9e20-9983e9b26b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:42.903213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 212/420] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:42.905048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c15ed9-5d43-44c8-9e20-9983e9b26b64 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:42.917298 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:42.943897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 209/421] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:42.948619 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9cd49-3528-4292-97a8-34fe3067fab4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:43.033693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c15ed9-5d43-44c8-9e20-9983e9b26b64 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:43.037054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:43.083067 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9cd49-3528-4292-97a8-34fe3067fab4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:43.084501 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc94f2a-f31f-4b76-be6f-e2e9f36acc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:43.085654 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc94f2a-f31f-4b76-be6f-e2e9f36acc66 None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:43.085935 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc94f2a-f31f-4b76-be6f-e2e9f36acc66 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:43.086205 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc94f2a-f31f-4b76-be6f-e2e9f36acc66 None admin] PATH_INFO: `/v3/projects/a9054cd3a81847fdaa8f38a717dc0938/groups/e90f520365d74e5ca16ea41ee94bb18a/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:43.089147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc94f2a-f31f-4b76-be6f-e2e9f36acc66 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9054cd3a81847fdaa8f38a717dc0938, group_id=e90f520365d74e5ca16ea41ee94bb18a, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:43.090599 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc94f2a-f31f-4b76-be6f-e2e9f36acc66 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:43.111012 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 213/422] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jul 14 20:20:42 2022] PUT /identity/v3/projects/a9054cd3a81847fdaa8f38a717dc0938/groups/e90f520365d74e5ca16ea41ee94bb18a/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:20:43.112340 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc94f2a-f31f-4b76-be6f-e2e9f36acc66 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:43.146645 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d2f34-9cb5-4765-aa52-b125b29af870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:43.147915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d2f34-9cb5-4765-aa52-b125b29af870 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:43.148434 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d2f34-9cb5-4765-aa52-b125b29af870 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:43.148938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d2f34-9cb5-4765-aa52-b125b29af870 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:43.167313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d2f34-9cb5-4765-aa52-b125b29af870 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:43.179756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-a49d2f34-9cb5-4765-aa52-b125b29af870 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:43.179756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:43.179756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:43.179756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:43.179756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:43.179756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:43.179756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:43.179756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:43.179756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:43.179756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:43.179756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:43.179756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:43.179756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:43.179756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:43.179756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:43.179756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:43.180866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:43.180866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:43.180866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:43.180866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:43.180866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:43.180866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:43.180866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:43.180866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:43.180866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:43.180866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:43.180866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:43.180866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:43.180866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:43.180866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:43.180866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:43.180866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:43.181690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:43.181690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:43.181690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:43.181690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:43.181690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:43.181690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:43.181690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:43.181690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:43.181690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:43.181690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:43.181690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:43.181690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:43.184726 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 210/423] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:20:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:43.191689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d2f34-9cb5-4765-aa52-b125b29af870 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:43.274281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:43.326830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc94f2a-f31f-4b76-be6f-e2e9f36acc66 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:43.375628 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddf2b8-270a-48dc-9366-9cfaf3479288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:43.377385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddf2b8-270a-48dc-9366-9cfaf3479288 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:43.377964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddf2b8-270a-48dc-9366-9cfaf3479288 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:43.378394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddf2b8-270a-48dc-9366-9cfaf3479288 None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:43.380494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ddf2b8-270a-48dc-9366-9cfaf3479288 None admin] RBAC: Authorizing `identity:get_project(project_id=930b378be44b4bb7a2009c72a424f1d6)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:43.382427 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ddf2b8-270a-48dc-9366-9cfaf3479288 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:43.385673 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 214/424] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:43 2022] GET /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6 => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:43.388325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d2f34-9cb5-4765-aa52-b125b29af870 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:43.400981 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddf2b8-270a-48dc-9366-9cfaf3479288 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:43.423824 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:43.448256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222dabd-05aa-4d06-94fa-8d0365aed925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:43.448942 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222dabd-05aa-4d06-94fa-8d0365aed925 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:43.449224 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222dabd-05aa-4d06-94fa-8d0365aed925 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:43.449503 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222dabd-05aa-4d06-94fa-8d0365aed925 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:43.450264 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222dabd-05aa-4d06-94fa-8d0365aed925 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:43.450572 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222dabd-05aa-4d06-94fa-8d0365aed925 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:43.452024 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222dabd-05aa-4d06-94fa-8d0365aed925 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:43.468197 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 211/425] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:20:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:43.479254 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222dabd-05aa-4d06-94fa-8d0365aed925 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:43.595230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddf2b8-270a-48dc-9366-9cfaf3479288 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:43.660830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535a6e7-7d63-4149-97eb-29e10b985b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:43.661703 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535a6e7-7d63-4149-97eb-29e10b985b4c None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:43.661825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222dabd-05aa-4d06-94fa-8d0365aed925 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:43.661986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535a6e7-7d63-4149-97eb-29e10b985b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:43.662258 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535a6e7-7d63-4149-97eb-29e10b985b4c None admin] PATH_INFO: `/v3/groups/6e193bbfa6174cbdb7b4484137920fb2` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:43.663490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535a6e7-7d63-4149-97eb-29e10b985b4c None admin] RBAC: Authorizing `identity:get_group(group_id=6e193bbfa6174cbdb7b4484137920fb2)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:43.664957 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535a6e7-7d63-4149-97eb-29e10b985b4c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:43.666917 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 215/426] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 14 20:20:43 2022] GET /identity/v3/groups/6e193bbfa6174cbdb7b4484137920fb2 => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:43.672796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfaefb-beae-4c67-9d7e-301d35b3c7fa None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:43.673092 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfaefb-beae-4c67-9d7e-301d35b3c7fa None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:43.673373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfaefb-beae-4c67-9d7e-301d35b3c7fa None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:43.674968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 216/427] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:43.677452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfaefb-beae-4c67-9d7e-301d35b3c7fa None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:43.681702 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:43.713850 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca7b85-8f0a-446c-aa75-a5e85b2c3949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:43.714507 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca7b85-8f0a-446c-aa75-a5e85b2c3949 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:43.714788 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca7b85-8f0a-446c-aa75-a5e85b2c3949 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:43.715088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca7b85-8f0a-446c-aa75-a5e85b2c3949 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:43.790032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:43.795256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfaefb-beae-4c67-9d7e-301d35b3c7fa None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:43.811270 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca7b85-8f0a-446c-aa75-a5e85b2c3949 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:43.813011 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca7b85-8f0a-446c-aa75-a5e85b2c3949 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:43.840067 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee7b22-a033-400a-951f-5d190ab569a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:43.840939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee7b22-a033-400a-951f-5d190ab569a2 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:43.841329 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee7b22-a033-400a-951f-5d190ab569a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:43.841707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee7b22-a033-400a-951f-5d190ab569a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:43.843357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee7b22-a033-400a-951f-5d190ab569a2 None admin] RBAC: Adding query filter params (group.id=6e193bbfa6174cbdb7b4484137920fb2, role.id=b778e530d54948c7bc31f5b9231ea288, scope.project.id=930b378be44b4bb7a2009c72a424f1d6) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:43.844052 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee7b22-a033-400a-951f-5d190ab569a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:43.846469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee7b22-a033-400a-951f-5d190ab569a2 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:43.859251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 217/428] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 14 20:20:43 2022] GET /identity/v3/role_assignments?group.id=6e193bbfa6174cbdb7b4484137920fb2&scope.project.id=930b378be44b4bb7a2009c72a424f1d6&role.id=b778e530d54948c7bc31f5b9231ea288 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:43.860891 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481810e3-d488-4e45-acbd-fa21079ef3f0 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:43.861334 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481810e3-d488-4e45-acbd-fa21079ef3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:43.861742 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481810e3-d488-4e45-acbd-fa21079ef3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:43.899144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-481810e3-d488-4e45-acbd-fa21079ef3f0 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:43.900380 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-481810e3-d488-4e45-acbd-fa21079ef3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:43.943553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 212/429] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:43.952922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca7b85-8f0a-446c-aa75-a5e85b2c3949 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:44.047839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:44.059838 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 218/430] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:44.088695 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481810e3-d488-4e45-acbd-fa21079ef3f0 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:44.117021 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:44.135007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca7b85-8f0a-446c-aa75-a5e85b2c3949 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:44.172744 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6e9bb-d122-4a91-baf1-c58e5fb89560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:44.174065 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6e9bb-d122-4a91-baf1-c58e5fb89560 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:44.174567 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6e9bb-d122-4a91-baf1-c58e5fb89560 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:44.175078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6e9bb-d122-4a91-baf1-c58e5fb89560 None admin] PATH_INFO: `/v3/projects/930b378be44b4bb7a2009c72a424f1d6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:44.176582 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6e9bb-d122-4a91-baf1-c58e5fb89560 None admin] RBAC: Authorizing `identity:get_project(project_id=930b378be44b4bb7a2009c72a424f1d6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:44.178389 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6e9bb-d122-4a91-baf1-c58e5fb89560 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:44.180486 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 213/431] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:43 2022] GET /identity/v3/projects/930b378be44b4bb7a2009c72a424f1d6 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:44.187024 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6e9bb-d122-4a91-baf1-c58e5fb89560 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:44.209408 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481810e3-d488-4e45-acbd-fa21079ef3f0 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:44.255863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8b379-4ed7-4e7e-9767-3d648a25bb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:44.256137 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8b379-4ed7-4e7e-9767-3d648a25bb2d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:44.256402 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8b379-4ed7-4e7e-9767-3d648a25bb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:44.256679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8b379-4ed7-4e7e-9767-3d648a25bb2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:44.269758 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8b379-4ed7-4e7e-9767-3d648a25bb2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:44.278261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-67f8b379-4ed7-4e7e-9767-3d648a25bb2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:44.278261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:44.278261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:44.278261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:44.278261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:44.278261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:44.278261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:44.278261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:44.278261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:44.278261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:44.278261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:44.278261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:44.278261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:44.278261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:44.278261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:44.278261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:44.279354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:44.279354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:44.279354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:44.279354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:44.279354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:44.279354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:44.279354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:44.279354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:44.279354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:44.279354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:44.279354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:44.279354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:44.279354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:44.279354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:44.279354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:44.279354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:44.285231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:44.285231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:44.285231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:44.285231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:44.285231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:44.285231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:44.285231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:44.285231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:44.285231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:44.285231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:44.285231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 20:20:44.285231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:44.285231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 219/432] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:44.289345 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8b379-4ed7-4e7e-9767-3d648a25bb2d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:44.356548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6e9bb-d122-4a91-baf1-c58e5fb89560 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:44.370431 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:44.394745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3d568-8090-4fd4-a6ab-b7438ef8fbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:44.395601 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3d568-8090-4fd4-a6ab-b7438ef8fbda None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:44.395882 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3d568-8090-4fd4-a6ab-b7438ef8fbda None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:44.396155 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3d568-8090-4fd4-a6ab-b7438ef8fbda None admin] PATH_INFO: `/v3/groups/6e193bbfa6174cbdb7b4484137920fb2` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:44.397339 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f3d568-8090-4fd4-a6ab-b7438ef8fbda None admin] RBAC: Authorizing `identity:get_group(group_id=6e193bbfa6174cbdb7b4484137920fb2)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:44.398787 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f3d568-8090-4fd4-a6ab-b7438ef8fbda None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:44.400673 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 214/433] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 14 20:20:44 2022] GET /identity/v3/groups/6e193bbfa6174cbdb7b4484137920fb2 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:44.401668 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453caa8-626f-4361-b2f3-70700120f293 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:44.401948 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453caa8-626f-4361-b2f3-70700120f293 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:44.402312 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453caa8-626f-4361-b2f3-70700120f293 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:44.403858 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 215/434] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:44.408333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453caa8-626f-4361-b2f3-70700120f293 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:44.418291 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8b379-4ed7-4e7e-9767-3d648a25bb2d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:44.434742 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:44.456349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207622a-7fbb-4431-9de8-54fcf7bd0df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:44.457020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207622a-7fbb-4431-9de8-54fcf7bd0df5 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:44.457372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207622a-7fbb-4431-9de8-54fcf7bd0df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:44.457638 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207622a-7fbb-4431-9de8-54fcf7bd0df5 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:44.458430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207622a-7fbb-4431-9de8-54fcf7bd0df5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:44.458728 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207622a-7fbb-4431-9de8-54fcf7bd0df5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:44.460176 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207622a-7fbb-4431-9de8-54fcf7bd0df5 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:44.470922 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 220/435] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:20:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:44.472254 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acc1e3-29c3-4cb4-b460-2baab81e057a None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:44.472551 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acc1e3-29c3-4cb4-b460-2baab81e057a None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:44.472834 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acc1e3-29c3-4cb4-b460-2baab81e057a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:44.490113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-14acc1e3-29c3-4cb4-b460-2baab81e057a None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:44.491380 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-14acc1e3-29c3-4cb4-b460-2baab81e057a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:44.527141 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453caa8-626f-4361-b2f3-70700120f293 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:44.581427 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4175310-2e65-40be-a6dd-17da1543628d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:44.582087 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4175310-2e65-40be-a6dd-17da1543628d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:44.582372 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4175310-2e65-40be-a6dd-17da1543628d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:44.582643 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4175310-2e65-40be-a6dd-17da1543628d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:44.583322 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 221/436] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:44.583509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4175310-2e65-40be-a6dd-17da1543628d None admin] RBAC: Adding query filter params (group.id=6e193bbfa6174cbdb7b4484137920fb2, role.id=6b2eb3e4cefd4a2db4f18e04a158600a, scope.project.id=930b378be44b4bb7a2009c72a424f1d6) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:44.583820 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4175310-2e65-40be-a6dd-17da1543628d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:44.585258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4175310-2e65-40be-a6dd-17da1543628d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:44.598621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 216/437] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 14 20:20:44 2022] GET /identity/v3/role_assignments?group.id=6e193bbfa6174cbdb7b4484137920fb2&scope.project.id=930b378be44b4bb7a2009c72a424f1d6&role.id=6b2eb3e4cefd4a2db4f18e04a158600a => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:44.598823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14acc1e3-29c3-4cb4-b460-2baab81e057a None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:44.607690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4175310-2e65-40be-a6dd-17da1543628d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:44.669071 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:44.722823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:44.732982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14acc1e3-29c3-4cb4-b460-2baab81e057a None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:44.744870 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4175310-2e65-40be-a6dd-17da1543628d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:44.779934 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1cf98-6825-47e1-9074-e17987cb0e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:44.781257 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1cf98-6825-47e1-9074-e17987cb0e22 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:44.781682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1cf98-6825-47e1-9074-e17987cb0e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:44.782366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1cf98-6825-47e1-9074-e17987cb0e22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:44.793721 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e580b-ef57-4927-b8a0-fd838e089bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:44.795988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e580b-ef57-4927-b8a0-fd838e089bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:44.795988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e580b-ef57-4927-b8a0-fd838e089bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:44.795988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e580b-ef57-4927-b8a0-fd838e089bf0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:44.808309 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e580b-ef57-4927-b8a0-fd838e089bf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:44.817568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-1d6e580b-ef57-4927-b8a0-fd838e089bf0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:44.817568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:44.817568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:44.817568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:44.817568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:44.817568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:44.817568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:44.817568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:44.817568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:44.817568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:44.817568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:44.817568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:44.817568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:44.817568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:44.817568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:44.817568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:44.818499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:44.818499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:44.818499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:44.818499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:44.818499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:44.818499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:44.818499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:44.818499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:44.818499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:44.818499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:44.818499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:44.818499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:44.818499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:44.818499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:44.818499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:44.818499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:44.819311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:44.819311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:44.819311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:44.819311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:44.819311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:44.819311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:44.819311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:44.819311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:44.819311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:44.819311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:44.819311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:44.819311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:44.819311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 217/438] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:20:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:44.824833 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e580b-ef57-4927-b8a0-fd838e089bf0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:44.879179 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1cf98-6825-47e1-9074-e17987cb0e22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:44.880640 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1cf98-6825-47e1-9074-e17987cb0e22 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:44.953577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:44.956287 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e580b-ef57-4927-b8a0-fd838e089bf0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:45.001222 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1577a5-d345-4c1b-91b8-a2f4f74d53e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:45.001959 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1577a5-d345-4c1b-91b8-a2f4f74d53e4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:45.002240 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1577a5-d345-4c1b-91b8-a2f4f74d53e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:45.002514 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1577a5-d345-4c1b-91b8-a2f4f74d53e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:45.003343 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1577a5-d345-4c1b-91b8-a2f4f74d53e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:45.003649 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1577a5-d345-4c1b-91b8-a2f4f74d53e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:45.005025 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1577a5-d345-4c1b-91b8-a2f4f74d53e4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:45.014729 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 218/439] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:20:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:45.021000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1577a5-d345-4c1b-91b8-a2f4f74d53e4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:45.024641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 222/440] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:45.030825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1cf98-6825-47e1-9074-e17987cb0e22 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:45.034623 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:45.157840 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1577a5-d345-4c1b-91b8-a2f4f74d53e4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:45.160569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1cf98-6825-47e1-9074-e17987cb0e22 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:45.198581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f49f5-3b92-46d6-9cde-1499466589c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:45.199271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f49f5-3b92-46d6-9cde-1499466589c7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:45.199375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1304a-35cb-414c-993f-83db43bece3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:45.199579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f49f5-3b92-46d6-9cde-1499466589c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:45.199823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f49f5-3b92-46d6-9cde-1499466589c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:45.200223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1304a-35cb-414c-993f-83db43bece3d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:45.200505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1304a-35cb-414c-993f-83db43bece3d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:45.200830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1304a-35cb-414c-993f-83db43bece3d None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:45.202572 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1304a-35cb-414c-993f-83db43bece3d None admin] RBAC: Authorizing `identity:get_project(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:45.208408 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1304a-35cb-414c-993f-83db43bece3d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:45.210353 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 223/441] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:45 2022] GET /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:45.215827 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1304a-35cb-414c-993f-83db43bece3d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:45.235923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:45.282556 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f49f5-3b92-46d6-9cde-1499466589c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:45.284181 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f49f5-3b92-46d6-9cde-1499466589c7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:45.294714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:45.336678 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1304a-35cb-414c-993f-83db43bece3d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:45.374493 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840389f-d2d9-4175-9981-a01661bee0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:45.375550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840389f-d2d9-4175-9981-a01661bee0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:45.376357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840389f-d2d9-4175-9981-a01661bee0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:45.376529 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840389f-d2d9-4175-9981-a01661bee0e9 None admin] PATH_INFO: `/v3/groups/6e193bbfa6174cbdb7b4484137920fb2` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:45.377988 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840389f-d2d9-4175-9981-a01661bee0e9 None admin] RBAC: Authorizing `identity:get_group(group_id=6e193bbfa6174cbdb7b4484137920fb2)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:45.379938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840389f-d2d9-4175-9981-a01661bee0e9 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:45.386242 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 224/442] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 14 20:20:45 2022] GET /identity/v3/groups/6e193bbfa6174cbdb7b4484137920fb2 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:45.387374 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840389f-d2d9-4175-9981-a01661bee0e9 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:45.395859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 219/443] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:45.401393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f49f5-3b92-46d6-9cde-1499466589c7 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:45.484998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:45.502636 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840389f-d2d9-4175-9981-a01661bee0e9 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:45.510393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f49f5-3b92-46d6-9cde-1499466589c7 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:45.519926 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:45.538755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e21f2-8486-4dbf-bd2a-7f12828976c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:45.539442 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e21f2-8486-4dbf-bd2a-7f12828976c7 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:45.539726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e21f2-8486-4dbf-bd2a-7f12828976c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:45.540002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e21f2-8486-4dbf-bd2a-7f12828976c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:45.540940 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e21f2-8486-4dbf-bd2a-7f12828976c7 None admin] RBAC: Adding query filter params (group.id=6e193bbfa6174cbdb7b4484137920fb2, role.id=b778e530d54948c7bc31f5b9231ea288, scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:45.541449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e21f2-8486-4dbf-bd2a-7f12828976c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:45.542687 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e21f2-8486-4dbf-bd2a-7f12828976c7 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:45.546380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfcfd2-e447-419a-9bbc-9d8987929095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:45.547269 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfcfd2-e447-419a-9bbc-9d8987929095 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:45.547549 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfcfd2-e447-419a-9bbc-9d8987929095 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:45.547823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfcfd2-e447-419a-9bbc-9d8987929095 None admin] PATH_INFO: `/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:45.549004 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfcfd2-e447-419a-9bbc-9d8987929095 None admin] RBAC: Authorizing `identity:get_project(project_id=3ae8ed2d5f0446129f0a26f4b142cdb5)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:45.549444 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 225/444] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 14 20:20:45 2022] GET /identity/v3/role_assignments?group.id=6e193bbfa6174cbdb7b4484137920fb2&scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5&role.id=b778e530d54948c7bc31f5b9231ea288 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:45.550423 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfcfd2-e447-419a-9bbc-9d8987929095 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:45.550700 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c6f07-341e-4178-86ca-f12c1b784631 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:45.551019 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c6f07-341e-4178-86ca-f12c1b784631 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:45.551389 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c6f07-341e-4178-86ca-f12c1b784631 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:45.557057 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 220/445] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:45 2022] GET /identity/v3/projects/3ae8ed2d5f0446129f0a26f4b142cdb5 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:45.557255 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 226/446] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:45.558568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23971c4-c254-43b8-a477-c96ab3cc37d5 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:45.558857 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23971c4-c254-43b8-a477-c96ab3cc37d5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:45.559170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23971c4-c254-43b8-a477-c96ab3cc37d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:45.564260 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c6f07-341e-4178-86ca-f12c1b784631 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:45.580431 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a23971c4-c254-43b8-a477-c96ab3cc37d5 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:45.581630 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a23971c4-c254-43b8-a477-c96ab3cc37d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:45.667509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 221/447] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:45.692838 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23971c4-c254-43b8-a477-c96ab3cc37d5 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:45.702961 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c6f07-341e-4178-86ca-f12c1b784631 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:45.745772 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26048d-af05-45c8-b07a-dfd96862d72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:45.746646 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26048d-af05-45c8-b07a-dfd96862d72f None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:45.746949 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26048d-af05-45c8-b07a-dfd96862d72f None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:45.747388 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26048d-af05-45c8-b07a-dfd96862d72f None admin] PATH_INFO: `/v3/groups/6e193bbfa6174cbdb7b4484137920fb2` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:45.749075 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26048d-af05-45c8-b07a-dfd96862d72f None admin] RBAC: Authorizing `identity:get_group(group_id=6e193bbfa6174cbdb7b4484137920fb2)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:45.750799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26048d-af05-45c8-b07a-dfd96862d72f None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:45.752467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 227/448] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 14 20:20:45 2022] GET /identity/v3/groups/6e193bbfa6174cbdb7b4484137920fb2 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:45.759047 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26048d-af05-45c8-b07a-dfd96862d72f None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:45.778823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:45.824266 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23971c4-c254-43b8-a477-c96ab3cc37d5 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:45.833501 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:45.871377 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666b1df-f85b-4bc6-a930-e49701e80ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:45.872179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666b1df-f85b-4bc6-a930-e49701e80ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:45.872468 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666b1df-f85b-4bc6-a930-e49701e80ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:45.873080 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666b1df-f85b-4bc6-a930-e49701e80ba2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:45.886719 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26048d-af05-45c8-b07a-dfd96862d72f None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:45.888429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666b1df-f85b-4bc6-a930-e49701e80ba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:45.896616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-e666b1df-f85b-4bc6-a930-e49701e80ba2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:20:45.896616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:45.896616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:45.896616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:45.896616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:45.896616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:45.896616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:45.896616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:45.896616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:45.896616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:45.896616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:45.896616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:45.896616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:45.896616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:45.896616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:45.896616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:45.897685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:45.897685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:45.897685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:45.897685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:45.897685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:45.897685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:45.897685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:45.897685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:45.897685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:45.897685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:45.897685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:45.897685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:45.897685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:45.897685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:45.897685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:45.897685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:45.898752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:45.898752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:45.898752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:45.898752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:45.898752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:45.898752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:45.898752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:45.898752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:45.898752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:45.898752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:45.898752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:20:45.898752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:45.898752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 222/449] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:20:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:45.903600 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666b1df-f85b-4bc6-a930-e49701e80ba2 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:45.932180 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7969c-58ca-4762-aac7-1a83a642657d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:45.932848 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7969c-58ca-4762-aac7-1a83a642657d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:45.933133 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7969c-58ca-4762-aac7-1a83a642657d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:45.933410 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7969c-58ca-4762-aac7-1a83a642657d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:45.934254 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7969c-58ca-4762-aac7-1a83a642657d None admin] RBAC: Adding query filter params (group.id=6e193bbfa6174cbdb7b4484137920fb2, role.id=6b2eb3e4cefd4a2db4f18e04a158600a, scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:45.934574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7969c-58ca-4762-aac7-1a83a642657d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:45.936083 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7969c-58ca-4762-aac7-1a83a642657d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:45.942148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 228/450] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 14 20:20:45 2022] GET /identity/v3/role_assignments?group.id=6e193bbfa6174cbdb7b4484137920fb2&scope.project.id=3ae8ed2d5f0446129f0a26f4b142cdb5&role.id=6b2eb3e4cefd4a2db4f18e04a158600a => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:46.025628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666b1df-f85b-4bc6-a930-e49701e80ba2 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:46.062075 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1d7c8-f4fe-44d1-a28c-4831ed96b8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:46.062757 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1d7c8-f4fe-44d1-a28c-4831ed96b8bc None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:46.063078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1d7c8-f4fe-44d1-a28c-4831ed96b8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:46.063357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1d7c8-f4fe-44d1-a28c-4831ed96b8bc None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:46.064122 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1d7c8-f4fe-44d1-a28c-4831ed96b8bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:46.064431 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1d7c8-f4fe-44d1-a28c-4831ed96b8bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:46.065836 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1d7c8-f4fe-44d1-a28c-4831ed96b8bc None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:46.076655 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 223/451] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:46.082490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7969c-58ca-4762-aac7-1a83a642657d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:46.173606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:46.192222 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7969c-58ca-4762-aac7-1a83a642657d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:46.228014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01713d18-1e4d-4fa3-9370-027c55de20ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:46.228682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01713d18-1e4d-4fa3-9370-027c55de20ca None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:46.228968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01713d18-1e4d-4fa3-9370-027c55de20ca None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:46.229248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01713d18-1e4d-4fa3-9370-027c55de20ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:46.297651 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01713d18-1e4d-4fa3-9370-027c55de20ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:46.299091 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01713d18-1e4d-4fa3-9370-027c55de20ca None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:46.378345 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:46.397511 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 229/452] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:46.403168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1d7c8-f4fe-44d1-a28c-4831ed96b8bc None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:46.431624 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:46.501220 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1d7c8-f4fe-44d1-a28c-4831ed96b8bc None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:46.535160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd540ff-df12-4df0-86a6-73d0408ba5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:46.536005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd540ff-df12-4df0-86a6-73d0408ba5fb None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:46.536286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd540ff-df12-4df0-86a6-73d0408ba5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:46.536563 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd540ff-df12-4df0-86a6-73d0408ba5fb None admin] PATH_INFO: `/v3/projects/a9054cd3a81847fdaa8f38a717dc0938` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:46.537728 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd540ff-df12-4df0-86a6-73d0408ba5fb None admin] RBAC: Authorizing `identity:get_project(project_id=a9054cd3a81847fdaa8f38a717dc0938)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:46.539196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd540ff-df12-4df0-86a6-73d0408ba5fb None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:46.540855 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 224/453] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:20:46 2022] GET /identity/v3/projects/a9054cd3a81847fdaa8f38a717dc0938 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:46.547319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01713d18-1e4d-4fa3-9370-027c55de20ca None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:46.655252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01713d18-1e4d-4fa3-9370-027c55de20ca None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:46.688714 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e710a-8600-4f8e-b47c-8a08f4d5691b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:46.689560 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e710a-8600-4f8e-b47c-8a08f4d5691b None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:46.689844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e710a-8600-4f8e-b47c-8a08f4d5691b None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:46.690118 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e710a-8600-4f8e-b47c-8a08f4d5691b None admin] PATH_INFO: `/v3/groups/e90f520365d74e5ca16ea41ee94bb18a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:46.691358 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e710a-8600-4f8e-b47c-8a08f4d5691b None admin] RBAC: Authorizing `identity:get_group(group_id=e90f520365d74e5ca16ea41ee94bb18a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:46.692786 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e710a-8600-4f8e-b47c-8a08f4d5691b None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:46.694421 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 230/454] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 14 20:20:46 2022] GET /identity/v3/groups/e90f520365d74e5ca16ea41ee94bb18a => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:46.700398 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd540ff-df12-4df0-86a6-73d0408ba5fb None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:46.792313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:46.798647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd540ff-df12-4df0-86a6-73d0408ba5fb None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:46.831671 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc7684-409a-46f2-bec0-fbdde5b4ec31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:46.832310 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc7684-409a-46f2-bec0-fbdde5b4ec31 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:46.832590 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc7684-409a-46f2-bec0-fbdde5b4ec31 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:46.832861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc7684-409a-46f2-bec0-fbdde5b4ec31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:46.833671 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc7684-409a-46f2-bec0-fbdde5b4ec31 None admin] RBAC: Adding query filter params (group.id=e90f520365d74e5ca16ea41ee94bb18a, role.id=5ca112885467422684e134cb0f94865a, scope.project.id=a9054cd3a81847fdaa8f38a717dc0938) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:46.833977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc7684-409a-46f2-bec0-fbdde5b4ec31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:46.835443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc7684-409a-46f2-bec0-fbdde5b4ec31 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:46.841078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 225/455] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 14 20:20:46 2022] GET /identity/v3/role_assignments?group.id=e90f520365d74e5ca16ea41ee94bb18a&scope.project.id=a9054cd3a81847fdaa8f38a717dc0938&role.id=5ca112885467422684e134cb0f94865a => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:49.460928 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8236555-94be-48b3-8f9b-c1989e058fd4 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:49.461479 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8236555-94be-48b3-8f9b-c1989e058fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:49.461479 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8236555-94be-48b3-8f9b-c1989e058fd4 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:49.463103 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 231/456] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:49.469073 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0630e8-ea38-4214-a05e-47bfdbdc4e30 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:49.469490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0630e8-ea38-4214-a05e-47bfdbdc4e30 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:49.469666 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0630e8-ea38-4214-a05e-47bfdbdc4e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:49.496242 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8f0630e8-ea38-4214-a05e-47bfdbdc4e30 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:49.497565 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8f0630e8-ea38-4214-a05e-47bfdbdc4e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:49.540666 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56262d-3f75-4993-868d-13637d9fd6ce None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:49.540969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56262d-3f75-4993-868d-13637d9fd6ce None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:49.541223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56262d-3f75-4993-868d-13637d9fd6ce None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:49.542858 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 232/457] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:20:49.548214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c1f87-54a9-48a5-8b8c-7e5273932b12 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:49.548504 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c1f87-54a9-48a5-8b8c-7e5273932b12 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:49.548782 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c1f87-54a9-48a5-8b8c-7e5273932b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:49.574442 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-212c1f87-54a9-48a5-8b8c-7e5273932b12 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:49.575681 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-212c1f87-54a9-48a5-8b8c-7e5273932b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:49.622034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 226/458] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:20:49.649688 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0630e8-ea38-4214-a05e-47bfdbdc4e30 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:49.671346 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:49.709806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 233/459] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:49.749059 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c1f87-54a9-48a5-8b8c-7e5273932b12 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:49.775444 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0630e8-ea38-4214-a05e-47bfdbdc4e30 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:49.785787 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:49.815028 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71594cd0-2f49-4bef-8930-702f0014cde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:49.815832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71594cd0-2f49-4bef-8930-702f0014cde2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:49.816113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71594cd0-2f49-4bef-8930-702f0014cde2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:49.816716 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71594cd0-2f49-4bef-8930-702f0014cde2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:49.827855 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71594cd0-2f49-4bef-8930-702f0014cde2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:49.829309 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71594cd0-2f49-4bef-8930-702f0014cde2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:49.849124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-71594cd0-2f49-4bef-8930-702f0014cde2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:49.849124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:49.849124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:49.849124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:49.849124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:49.849124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:49.849124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:49.849124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:49.849124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:49.849124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:49.849124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:49.849124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:49.849124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:20:49.849124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:20:49.849124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:20:49.850046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:20:49.850046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:49.850046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:49.850046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:49.850046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:49.850046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:49.850046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:49.850046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:49.850046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:49.850046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:49.850046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:49.850046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:49.850046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:49.850046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:49.850046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:49.850046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:49.850046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:49.850885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:20:49.850885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:20:49.850885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:49.850885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:49.850885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 227/460] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:49.851626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921699d9-bf5f-4667-8be1-21fc05d9b1f9 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:49.851626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921699d9-bf5f-4667-8be1-21fc05d9b1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:49.851867 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921699d9-bf5f-4667-8be1-21fc05d9b1f9 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:49.857112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 228/461] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:49.857112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921699d9-bf5f-4667-8be1-21fc05d9b1f9 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:49.890162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c1f87-54a9-48a5-8b8c-7e5273932b12 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:49.929497 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e5157-c24d-44cc-81f2-25b72d2a2203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:49.930643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e5157-c24d-44cc-81f2-25b72d2a2203 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:49.931205 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e5157-c24d-44cc-81f2-25b72d2a2203 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:49.931706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e5157-c24d-44cc-81f2-25b72d2a2203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:49.943773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e5157-c24d-44cc-81f2-25b72d2a2203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:49.945783 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e5157-c24d-44cc-81f2-25b72d2a2203 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:49.964790 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921699d9-bf5f-4667-8be1-21fc05d9b1f9 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:49.965017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-0e0e5157-c24d-44cc-81f2-25b72d2a2203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:49.965017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:49.965017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:49.965017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:49.965017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:49.965017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:49.965017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:49.965017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:49.965017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:49.965017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:49.965017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:49.965017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:49.965017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:20:49.965017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:20:49.965017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:20:49.965799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:20:49.965799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:49.965799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:49.965799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:49.965799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:49.965799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:49.965799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:49.965799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:49.965799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:49.965799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:49.965799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:49.965799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:49.965799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:49.965799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:49.965799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:49.965799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:49.965799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:49.967116 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:20:49.967116 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:20:49.967116 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:49.967116 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:49.967116 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 234/462] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:49.968136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9aa13-7224-4e35-bd0a-ccdccdfb7626 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:49.968489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9aa13-7224-4e35-bd0a-ccdccdfb7626 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:49.968833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9aa13-7224-4e35-bd0a-ccdccdfb7626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:49.985827 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-9ff9aa13-7224-4e35-bd0a-ccdccdfb7626 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:49.987053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9ff9aa13-7224-4e35-bd0a-ccdccdfb7626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:50.004253 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f288d-cdaa-45d6-b3c4-0b2186c59ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:50.004950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f288d-cdaa-45d6-b3c4-0b2186c59ecc None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:50.005235 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f288d-cdaa-45d6-b3c4-0b2186c59ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:50.005629 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f288d-cdaa-45d6-b3c4-0b2186c59ecc None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:50.006362 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f288d-cdaa-45d6-b3c4-0b2186c59ecc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:50.006670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f288d-cdaa-45d6-b3c4-0b2186c59ecc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:50.008100 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f288d-cdaa-45d6-b3c4-0b2186c59ecc None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:50.014314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:50.031274 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 229/463] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:50.033328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e8a3b-6d99-4dec-88f8-946a2d394e27 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:50.033621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e8a3b-6d99-4dec-88f8-946a2d394e27 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:50.033897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e8a3b-6d99-4dec-88f8-946a2d394e27 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:50.035681 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 230/464] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:50.036239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e8a3b-6d99-4dec-88f8-946a2d394e27 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:50.040684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:50.063457 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 235/465] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:50.064093 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9aa13-7224-4e35-bd0a-ccdccdfb7626 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:50.163848 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e8a3b-6d99-4dec-88f8-946a2d394e27 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:50.171102 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9aa13-7224-4e35-bd0a-ccdccdfb7626 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:50.203177 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1267a9-1bb1-421e-b872-93892b77175a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:50.203955 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1267a9-1bb1-421e-b872-93892b77175a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:50.204243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1267a9-1bb1-421e-b872-93892b77175a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:50.204518 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1267a9-1bb1-421e-b872-93892b77175a None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:50.205248 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1267a9-1bb1-421e-b872-93892b77175a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:50.205557 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1267a9-1bb1-421e-b872-93892b77175a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:50.207012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1267a9-1bb1-421e-b872-93892b77175a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:50.208639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a8d85-aaaf-4a48-b8b0-b99ecf468ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:50.209440 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a8d85-aaaf-4a48-b8b0-b99ecf468ea1 None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:50.209726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a8d85-aaaf-4a48-b8b0-b99ecf468ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:50.209998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a8d85-aaaf-4a48-b8b0-b99ecf468ea1 None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:50.210704 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a8d85-aaaf-4a48-b8b0-b99ecf468ea1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:50.212198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a8d85-aaaf-4a48-b8b0-b99ecf468ea1 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:50.220713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 231/466] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:50.221986 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f51d2-0453-4a90-a577-1d2e886d6296 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:50.222277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f51d2-0453-4a90-a577-1d2e886d6296 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:50.222580 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f51d2-0453-4a90-a577-1d2e886d6296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:50.239595 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 236/467] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 14 20:20:50 2022] POST /identity/v3/users => generated 246 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:20:50.240533 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a8d85-aaaf-4a48-b8b0-b99ecf468ea1 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:50.251725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ea5f51d2-0453-4a90-a577-1d2e886d6296 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:50.253355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ea5f51d2-0453-4a90-a577-1d2e886d6296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:50.265225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:50.345228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:50.360379 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 232/468] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:50.362050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f51d2-0453-4a90-a577-1d2e886d6296 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:50.387203 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a8d85-aaaf-4a48-b8b0-b99ecf468ea1 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:50.440849 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc3512-98f1-45d6-9abb-15a3170a6f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:50.441672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc3512-98f1-45d6-9abb-15a3170a6f9e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:50.441962 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc3512-98f1-45d6-9abb-15a3170a6f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:50.442242 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc3512-98f1-45d6-9abb-15a3170a6f9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:50.455174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc3512-98f1-45d6-9abb-15a3170a6f9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:50.456638 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc3512-98f1-45d6-9abb-15a3170a6f9e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:50.478028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-06dc3512-98f1-45d6-9abb-15a3170a6f9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:50.478028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:50.478028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:50.478028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:50.478028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:50.478028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:50.478028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:50.478028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:50.478028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:50.478028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:50.478028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:50.478028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:50.478028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:20:50.478028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:20:50.478028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:20:50.479462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:20:50.479462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:50.479462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:50.479462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:50.479462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:50.479462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:50.479462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:50.479462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:50.479462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:50.479462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:50.479462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:50.479462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:50.479462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:50.479462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:50.479462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:50.479462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:50.479462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:50.480328 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:20:50.480328 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:20:50.480328 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:50.480328 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:50.480328 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 237/469] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:50.480605 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc3512-98f1-45d6-9abb-15a3170a6f9e None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:50.492873 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f51d2-0453-4a90-a577-1d2e886d6296 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:50.535189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16b365-b271-433f-80ee-0c117ea3fff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:50.535787 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16b365-b271-433f-80ee-0c117ea3fff7 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:50.536070 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16b365-b271-433f-80ee-0c117ea3fff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:50.536537 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16b365-b271-433f-80ee-0c117ea3fff7 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:50.537461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16b365-b271-433f-80ee-0c117ea3fff7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:50.539023 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16b365-b271-433f-80ee-0c117ea3fff7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:50.563311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 233/470] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 14 20:20:50 2022] POST /identity/v3/users => generated 248 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:20:50.571605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16b365-b271-433f-80ee-0c117ea3fff7 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:50.594060 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:50.679358 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:50.685320 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc3512-98f1-45d6-9abb-15a3170a6f9e None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:50.733994 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16b365-b271-433f-80ee-0c117ea3fff7 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:50.773480 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f38f4-c4c6-468c-a354-16212437bf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:50.774747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f38f4-c4c6-468c-a354-16212437bf15 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:50.776601 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e63e43-2891-4024-8181-330411e396d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:50.776601 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e63e43-2891-4024-8181-330411e396d4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:50.776601 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e63e43-2891-4024-8181-330411e396d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:50.776601 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e63e43-2891-4024-8181-330411e396d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:50.777196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e63e43-2891-4024-8181-330411e396d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:50.777660 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e63e43-2891-4024-8181-330411e396d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:50.778074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f38f4-c4c6-468c-a354-16212437bf15 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:50.778580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f38f4-c4c6-468c-a354-16212437bf15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:50.779111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e63e43-2891-4024-8181-330411e396d4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:50.794225 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 234/471] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:50.796026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddef5b1-c56e-4d52-8a11-29aa8eed11ef None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:50.796318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddef5b1-c56e-4d52-8a11-29aa8eed11ef None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:50.796649 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddef5b1-c56e-4d52-8a11-29aa8eed11ef None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:50.798228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 235/472] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:50.799864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddef5b1-c56e-4d52-8a11-29aa8eed11ef None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:50.801142 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f38f4-c4c6-468c-a354-16212437bf15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:50.802816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f38f4-c4c6-468c-a354-16212437bf15 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:50.824356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-ad8f38f4-c4c6-468c-a354-16212437bf15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:50.824356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:50.824356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:50.824356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:50.824356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:50.824356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:50.824356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:50.824356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:50.824356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:50.824356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:50.824356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:50.824356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:50.824356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:20:50.824356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:20:50.824356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:20:50.825284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:20:50.825284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:50.825284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:50.825284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:50.825284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:50.825284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:50.825284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:50.825284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:50.825284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:50.825284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:50.825284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:50.825284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:50.825284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:50.825284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:50.825284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:50.825284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:50.825284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:50.826228 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:20:50.826228 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:20:50.826228 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:50.826228 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:50.826228 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 238/473] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:50.827316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be5660-cb8b-4272-924d-f5db925378cf None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:50.827608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be5660-cb8b-4272-924d-f5db925378cf None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:50.827936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be5660-cb8b-4272-924d-f5db925378cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:50.846755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-89be5660-cb8b-4272-924d-f5db925378cf None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:50.848145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-89be5660-cb8b-4272-924d-f5db925378cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:50.936155 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddef5b1-c56e-4d52-8a11-29aa8eed11ef None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:50.940047 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 239/474] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:50.940984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be5660-cb8b-4272-924d-f5db925378cf None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:50.947979 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:50.984674 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce564e0f-7c3d-424b-b687-a753e4ccc2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:50.985634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce564e0f-7c3d-424b-b687-a753e4ccc2e3 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:50.986063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce564e0f-7c3d-424b-b687-a753e4ccc2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:50.986524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce564e0f-7c3d-424b-b687-a753e4ccc2e3 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:50.987408 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce564e0f-7c3d-424b-b687-a753e4ccc2e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:50.989045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce564e0f-7c3d-424b-b687-a753e4ccc2e3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:51.008596 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:51.020740 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 236/475] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 14 20:20:50 2022] POST /identity/v3/users => generated 248 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:20:51.022088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce564e0f-7c3d-424b-b687-a753e4ccc2e3 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:51.113398 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be5660-cb8b-4272-924d-f5db925378cf None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:51.195869 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c739f1-61bc-450a-b944-d5d03794df1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:51.196841 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c739f1-61bc-450a-b944-d5d03794df1b None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:51.197270 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c739f1-61bc-450a-b944-d5d03794df1b None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:51.197875 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c739f1-61bc-450a-b944-d5d03794df1b None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:51.200230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c739f1-61bc-450a-b944-d5d03794df1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:51.200725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c739f1-61bc-450a-b944-d5d03794df1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:51.203369 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c739f1-61bc-450a-b944-d5d03794df1b None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:51.226262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 240/476] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:51.231460 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce564e0f-7c3d-424b-b687-a753e4ccc2e3 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:51.233156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c739f1-61bc-450a-b944-d5d03794df1b None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:51.273340 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabf857-a026-47e0-bf11-2f0215ea4c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:51.273340 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabf857-a026-47e0-bf11-2f0215ea4c68 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:51.273664 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabf857-a026-47e0-bf11-2f0215ea4c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:51.273891 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabf857-a026-47e0-bf11-2f0215ea4c68 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:51.274645 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabf857-a026-47e0-bf11-2f0215ea4c68 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:51.276081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabf857-a026-47e0-bf11-2f0215ea4c68 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:51.284083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.sql.core [None req-2aabf857-a026-47e0-bf11-2f0215ea4c68 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 20:20:51.284083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 20:20:51.284083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [parameters: {'id': 'c2f13eb3d68b4873a724046dc0763611', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 20:20:51.284083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80177) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 20:20:51.303922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-2aabf857-a026-47e0-bf11-2f0215ea4c68 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 20:20:51.303922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:51.303922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 20:20:51.303922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 20:20:51.303922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 20:20:51.303922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 20:20:51.303922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 20:20:51.303922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 20:20:51.303922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 20:20:51.303922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application conn.query(q) Jul 14 20:20:51.303922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 20:20:51.303922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 20:20:51.303922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 20:20:51.303922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application result.read() Jul 14 20:20:51.303922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 20:20:51.304968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 20:20:51.304968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 20:20:51.304968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 20:20:51.304968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 20:20:51.304968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 20:20:51.304968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 20:20:51.304968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 20:20:51.304968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 20:20:51.304968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:51.304968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 20:20:51.304968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:51.304968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:51.304968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 20:20:51.304968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 20:20:51.304968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 14 20:20:51.304968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return ref.to_dict() Jul 14 20:20:51.304968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 20:20:51.304968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application next(self.gen) Jul 14 20:20:51.305865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 20:20:51.305865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application yield resource Jul 14 20:20:51.305865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 20:20:51.305865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application next(self.gen) Jul 14 20:20:51.305865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 20:20:51.305865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 20:20:51.305865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 20:20:51.305865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 20:20:51.305865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 20:20:51.305865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise self.value Jul 14 20:20:51.305865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 20:20:51.305865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 20:20:51.305865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 20:20:51.305865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application session.commit() Jul 14 20:20:51.305865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 20:20:51.305865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 20:20:51.305865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 20:20:51.306725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 20:20:51.306725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 20:20:51.306725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self.session.flush() Jul 14 20:20:51.306725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 20:20:51.306725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 20:20:51.306725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 20:20:51.306725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 20:20:51.306725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 20:20:51.306725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application compat.raise_( Jul 14 20:20:51.306725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 20:20:51.306725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception Jul 14 20:20:51.306725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 20:20:51.306725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 20:20:51.306725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 20:20:51.306725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 20:20:51.306725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 20:20:51.306725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 20:20:51.308916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 20:20:51.308916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 20:20:51.308916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 20:20:51.308916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 20:20:51.308916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 20:20:51.308916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 20:20:51.308916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 20:20:51.308916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 20:20:51.308916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 20:20:51.308916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 20:20:51.308916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 20:20:51.308916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 20:20:51.308916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 20:20:51.308916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 20:20:51.308916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 20:20:51.308916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception Jul 14 20:20:51.311794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 20:20:51.311794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 20:20:51.311794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 20:20:51.311794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 20:20:51.311794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 20:20:51.311794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 20:20:51.311794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 20:20:51.311794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application conn.query(q) Jul 14 20:20:51.311794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 20:20:51.311794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 20:20:51.311794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 20:20:51.311794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application result.read() Jul 14 20:20:51.311794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 20:20:51.311794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 20:20:51.311794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 20:20:51.311794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 20:20:51.313895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 20:20:51.313895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 20:20:51.313895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 20:20:51.313895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 20:20:51.313895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 20:20:51.313895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 20:20:51.313895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [parameters: {'id': 'c2f13eb3d68b4873a724046dc0763611', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 20:20:51.313895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 20:20:51.313895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:51.313895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:20:51.313895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:51.313895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:51.313895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:51.313895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:51.313895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:51.313895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:51.315890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:51.315890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:51.315890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:51.315890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:51.315890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:51.315890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:51.315890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 14 20:20:51.315890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 14 20:20:51.315890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:51.315890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:51.315890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 14 20:20:51.315890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 14 20:20:51.315890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 20:20:51.315890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 20:20:51.315890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 20:20:51.315890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:51.317858 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 237/477] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jul 14 20:20:51 2022] POST /identity/v3/roles => generated 144 bytes in 284 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:51.317858 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabf857-a026-47e0-bf11-2f0215ea4c68 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:51.346592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:51.359135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c739f1-61bc-450a-b944-d5d03794df1b None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:51.373351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:51.397694 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a928f-0942-494d-b619-1b30b8104d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:51.398737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a928f-0942-494d-b619-1b30b8104d25 None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:51.399249 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a928f-0942-494d-b619-1b30b8104d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:51.399722 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a928f-0942-494d-b619-1b30b8104d25 None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:51.400679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a928f-0942-494d-b619-1b30b8104d25 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:51.402349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a928f-0942-494d-b619-1b30b8104d25 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:51.425341 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 241/478] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 14 20:20:51 2022] POST /identity/v3/users => generated 249 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:20:51.442851 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabf857-a026-47e0-bf11-2f0215ea4c68 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:51.484160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52226ffa-0528-4adc-9722-d156cdf594e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:51.485008 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52226ffa-0528-4adc-9722-d156cdf594e9 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:51.485294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52226ffa-0528-4adc-9722-d156cdf594e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:51.485571 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52226ffa-0528-4adc-9722-d156cdf594e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:51.498407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52226ffa-0528-4adc-9722-d156cdf594e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:51.507056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-52226ffa-0528-4adc-9722-d156cdf594e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:51.507056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:51.507056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:51.507056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:51.507056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:51.507056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:51.507056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:51.507056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:51.507056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:51.507056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:51.507056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:51.507056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:51.507056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:51.507056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:51.507056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:51.507056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:51.508078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:51.508078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:51.508078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:51.508078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:51.508078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:51.508078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:51.508078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:51.508078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:51.508078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:51.508078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:51.508078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:51.508078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:51.508078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:51.508078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:51.508078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:51.508078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:51.508864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:51.508864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:51.508864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:51.508864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:51.508864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:51.508864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:51.508864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:51.508864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:51.508864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:51.508864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:51.508864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 20:20:51.508864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:51.508864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 238/479] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:20:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:51.514820 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a928f-0942-494d-b619-1b30b8104d25 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:51.642975 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a928f-0942-494d-b619-1b30b8104d25 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:51.674410 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0ca1f-d76c-492f-9d10-06fee1932281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:51.675122 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0ca1f-d76c-492f-9d10-06fee1932281 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:51.675412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0ca1f-d76c-492f-9d10-06fee1932281 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:51.675692 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0ca1f-d76c-492f-9d10-06fee1932281 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:51.676455 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0ca1f-d76c-492f-9d10-06fee1932281 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:51.676768 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0ca1f-d76c-492f-9d10-06fee1932281 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:51.678235 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0ca1f-d76c-492f-9d10-06fee1932281 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:51.687506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 242/480] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:20:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:52.538071 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464134c-cce5-47d3-a512-18c32d2caa0d None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:52.538579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464134c-cce5-47d3-a512-18c32d2caa0d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:52.538678 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464134c-cce5-47d3-a512-18c32d2caa0d None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:52.540483 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 239/481] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:52.547876 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45efba9-0fbf-47e8-93c4-b94ddd2a4ce0 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:52.548367 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45efba9-0fbf-47e8-93c4-b94ddd2a4ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:52.548560 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45efba9-0fbf-47e8-93c4-b94ddd2a4ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:52.576393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-a45efba9-0fbf-47e8-93c4-b94ddd2a4ce0 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:52.577606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-a45efba9-0fbf-47e8-93c4-b94ddd2a4ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:52.633478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:52.741744 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 243/482] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:52.788063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464134c-cce5-47d3-a512-18c32d2caa0d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:53.015655 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:53.043928 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464134c-cce5-47d3-a512-18c32d2caa0d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:53.093195 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c233124-7c0f-4c7c-8531-7f951040061b None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:53.093500 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c233124-7c0f-4c7c-8531-7f951040061b None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:53.094489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c233124-7c0f-4c7c-8531-7f951040061b None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:53.099542 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 244/483] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:53.104766 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5623a9-1d4b-49e0-85e1-b73faaeb39ec None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:53.105060 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5623a9-1d4b-49e0-85e1-b73faaeb39ec None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:53.105344 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5623a9-1d4b-49e0-85e1-b73faaeb39ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:53.106042 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84554e-0713-47db-9361-64ac079f7e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:53.107262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84554e-0713-47db-9361-64ac079f7e30 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:53.107749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84554e-0713-47db-9361-64ac079f7e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:53.108309 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84554e-0713-47db-9361-64ac079f7e30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:53.128988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce84554e-0713-47db-9361-64ac079f7e30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:53.137521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-ce84554e-0713-47db-9361-64ac079f7e30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:20:53.137521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:53.137521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:53.137521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:53.137521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:53.137521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:53.137521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:53.137521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:53.137521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:53.137521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:53.137521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:53.137521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:53.137521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:53.137521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:53.137521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:53.137521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:53.139074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:53.139074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:53.139074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:53.139074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:53.139074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:53.139074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:53.139074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:53.139074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:53.139074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:53.139074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:53.139074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:53.139074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:53.139074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:53.139074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:53.139074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:53.139074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:53.139871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:53.139871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:53.139871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:53.139871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:53.139871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:53.139871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:53.139871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:53.139871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:53.139871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:53.139871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:53.139871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:20:53.139871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:53.148165 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-8c5623a9-1d4b-49e0-85e1-b73faaeb39ec None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:53.148374 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 240/484] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:20:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:53.149476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-8c5623a9-1d4b-49e0-85e1-b73faaeb39ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:53.153988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84554e-0713-47db-9361-64ac079f7e30 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:53.302878 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84554e-0713-47db-9361-64ac079f7e30 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:53.340617 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ae0db-6894-4f00-8506-2cb959168d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:53.341732 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ae0db-6894-4f00-8506-2cb959168d0e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:53.342220 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ae0db-6894-4f00-8506-2cb959168d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:53.342696 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ae0db-6894-4f00-8506-2cb959168d0e None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:53.343677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ae0db-6894-4f00-8506-2cb959168d0e None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:53.344189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ae0db-6894-4f00-8506-2cb959168d0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:53.345807 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ae0db-6894-4f00-8506-2cb959168d0e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:53.355089 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 241/485] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:20:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:53.362618 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ae0db-6894-4f00-8506-2cb959168d0e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:53.369030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 245/486] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:53.377329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:53.428386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5623a9-1d4b-49e0-85e1-b73faaeb39ec None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:53.477635 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ae0db-6894-4f00-8506-2cb959168d0e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:53.514679 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d9939-f45d-4f40-bc90-096778c044b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:53.515850 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d9939-f45d-4f40-bc90-096778c044b5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:53.516338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d9939-f45d-4f40-bc90-096778c044b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:53.516838 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d9939-f45d-4f40-bc90-096778c044b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:53.528731 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d9939-f45d-4f40-bc90-096778c044b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:53.530521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d9939-f45d-4f40-bc90-096778c044b5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:53.548805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-804d9939-f45d-4f40-bc90-096778c044b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:53.548805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:53.548805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:53.548805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:53.548805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:53.548805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:53.548805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:53.548805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:53.548805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:53.548805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:53.548805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:53.548805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:53.548805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:20:53.548805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:20:53.548805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:20:53.550116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:20:53.550116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:53.550116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:53.550116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:53.550116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:53.550116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:53.550116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:53.550116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:53.550116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:53.550116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:53.550116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:53.550116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:53.550116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:53.550116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:53.550116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:53.550116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:53.550116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:53.550967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:20:53.550967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:20:53.550967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:53.550967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:53.553267 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 242/487] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:53.559901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d9939-f45d-4f40-bc90-096778c044b5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:53.626856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5623a9-1d4b-49e0-85e1-b73faaeb39ec None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:53.635080 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:53.639412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:53.679139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d9939-f45d-4f40-bc90-096778c044b5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:53.711182 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f6918-858e-41bc-870d-447e301ce82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:53.712004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f6918-858e-41bc-870d-447e301ce82d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:53.712292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f6918-858e-41bc-870d-447e301ce82d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:53.712570 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f6918-858e-41bc-870d-447e301ce82d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:53.716460 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4e6e2-b86a-43b2-a5ba-479e198bb490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:53.717163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4e6e2-b86a-43b2-a5ba-479e198bb490 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:53.717531 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4e6e2-b86a-43b2-a5ba-479e198bb490 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:53.717817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4e6e2-b86a-43b2-a5ba-479e198bb490 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:53.718560 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4e6e2-b86a-43b2-a5ba-479e198bb490 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:53.718870 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4e6e2-b86a-43b2-a5ba-479e198bb490 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:53.720409 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4e6e2-b86a-43b2-a5ba-479e198bb490 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:53.734345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 243/488] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:53.741130 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f6918-858e-41bc-870d-447e301ce82d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:53.743911 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4e6e2-b86a-43b2-a5ba-479e198bb490 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:53.749773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-4b9f6918-858e-41bc-870d-447e301ce82d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:20:53.749773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:53.749773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:53.749773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:53.749773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:53.749773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:53.749773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:53.749773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:53.749773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:53.749773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:53.749773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:53.749773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:53.749773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:53.749773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:53.749773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:53.749773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:53.750667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:53.750667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:53.750667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:53.750667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:53.750667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:53.750667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:53.750667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:53.750667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:53.750667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:53.750667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:53.750667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:53.750667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:53.750667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:53.750667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:53.750667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:53.750667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:53.751813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:53.751813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:53.751813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:53.751813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:53.751813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:53.751813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:53.751813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:53.751813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:53.751813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:53.751813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:53.751813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:20:53.751813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:53.751813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 246/489] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:20:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:53.757057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f6918-858e-41bc-870d-447e301ce82d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:53.940792 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4e6e2-b86a-43b2-a5ba-479e198bb490 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:53.946080 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f6918-858e-41bc-870d-447e301ce82d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:53.967769 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:53.981901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393d2ae-7a0e-41b7-ac9c-44a54c7c2b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:53.983691 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393d2ae-7a0e-41b7-ac9c-44a54c7c2b43 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:53.984146 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393d2ae-7a0e-41b7-ac9c-44a54c7c2b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:53.984559 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393d2ae-7a0e-41b7-ac9c-44a54c7c2b43 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:53.989248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f91bd-4284-4a4d-8475-6868dea02a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:53.989927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f91bd-4284-4a4d-8475-6868dea02a5c None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:53.990214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f91bd-4284-4a4d-8475-6868dea02a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:53.990491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f91bd-4284-4a4d-8475-6868dea02a5c None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:53.991283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f91bd-4284-4a4d-8475-6868dea02a5c None admin] RBAC: Adding query filter params (name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:53.991594 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f91bd-4284-4a4d-8475-6868dea02a5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:53.993151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f91bd-4284-4a4d-8475-6868dea02a5c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:53.994081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393d2ae-7a0e-41b7-ac9c-44a54c7c2b43 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:53.996067 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393d2ae-7a0e-41b7-ac9c-44a54c7c2b43 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:54.008014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 247/490] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:20:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:54.008014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6321a60-eedb-422c-9bcb-909194b1ce1e None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:54.008014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6321a60-eedb-422c-9bcb-909194b1ce1e None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:54.008014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6321a60-eedb-422c-9bcb-909194b1ce1e None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:54.008014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 248/491] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:54.008984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6321a60-eedb-422c-9bcb-909194b1ce1e None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:54.020964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-f393d2ae-7a0e-41b7-ac9c-44a54c7c2b43 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 20:20:54.020964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:54.020964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:54.020964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:54.020964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:54.020964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:54.020964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:54.020964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:54.020964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:54.020964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:54.020964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:54.020964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:54.020964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:20:54.020964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:20:54.020964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:20:54.020964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:20:54.021915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:54.021915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:54.021915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:20:54.021915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:20:54.021915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:20:54.021915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:20:54.021915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:54.021915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:54.021915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:54.021915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:54.021915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:54.021915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:54.021915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:54.021915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:54.021915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:54.021915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:54.021915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:54.021915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:54.022817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:54.022817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:54.022817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:20:54.022817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:20:54.022817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:20:54.022817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:20:54.022817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:20:54.022817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:20:54.022817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 20:20:54.022817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:54.025198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:54.031370 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 244/492] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 14 20:20:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:54.033297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1bebc-4b3d-4c63-838f-8766970be68c None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:54.033784 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1bebc-4b3d-4c63-838f-8766970be68c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:54.034271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1bebc-4b3d-4c63-838f-8766970be68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:54.054180 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-11a1bebc-4b3d-4c63-838f-8766970be68c None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:54.055869 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-11a1bebc-4b3d-4c63-838f-8766970be68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:54.133077 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 245/493] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:54.134527 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1bebc-4b3d-4c63-838f-8766970be68c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:54.135267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6321a60-eedb-422c-9bcb-909194b1ce1e None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:54.178223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c3d99-1666-4dd5-98df-42604564341d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:54.179536 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c3d99-1666-4dd5-98df-42604564341d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:54.180066 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c3d99-1666-4dd5-98df-42604564341d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:54.180547 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c3d99-1666-4dd5-98df-42604564341d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:54.193246 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c3d99-1666-4dd5-98df-42604564341d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:54.195848 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c3d99-1666-4dd5-98df-42604564341d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:54.218151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-205c3d99-1666-4dd5-98df-42604564341d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:54.218151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:54.218151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:54.218151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:54.218151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:54.218151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:54.218151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:54.218151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:54.218151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:54.218151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:54.218151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:54.218151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:54.218151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:20:54.218151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:20:54.218151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:20:54.219248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:20:54.219248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:54.219248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:54.219248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:54.219248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:54.219248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:54.219248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:54.219248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:54.219248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:54.219248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:54.219248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:54.219248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:54.219248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:54.219248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:54.219248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:54.219248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:54.219248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:54.220111 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:20:54.220111 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:20:54.220111 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:54.220111 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:54.225203 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 249/494] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:54.225203 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c3d99-1666-4dd5-98df-42604564341d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:54.252845 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1bebc-4b3d-4c63-838f-8766970be68c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:54.272581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:54.299044 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:54.299448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e30182-480c-4277-9477-46a373dd3f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:54.300139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e30182-480c-4277-9477-46a373dd3f11 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:54.300761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e30182-480c-4277-9477-46a373dd3f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:54.300761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e30182-480c-4277-9477-46a373dd3f11 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:54.301621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e30182-480c-4277-9477-46a373dd3f11 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:54.301964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e30182-480c-4277-9477-46a373dd3f11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:54.303585 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e30182-480c-4277-9477-46a373dd3f11 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:54.340488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c3d99-1666-4dd5-98df-42604564341d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:54.342600 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 246/495] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 14 20:20:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:54.343796 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0490bd9-5d92-48d0-85cf-6eee06a343db None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:54.344091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0490bd9-5d92-48d0-85cf-6eee06a343db None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:54.344372 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0490bd9-5d92-48d0-85cf-6eee06a343db None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:54.349864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 247/496] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:54.350492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0490bd9-5d92-48d0-85cf-6eee06a343db None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:54.384280 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef547924-4b10-41ef-8cf7-695aed677ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:54.385115 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef547924-4b10-41ef-8cf7-695aed677ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:54.385695 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef547924-4b10-41ef-8cf7-695aed677ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:54.385802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef547924-4b10-41ef-8cf7-695aed677ab7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:54.397940 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef547924-4b10-41ef-8cf7-695aed677ab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:54.406309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-ef547924-4b10-41ef-8cf7-695aed677ab7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:20:54.406309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:54.406309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:54.406309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:54.406309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:54.406309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:54.406309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:54.406309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:54.406309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:54.406309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:54.406309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:54.406309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:54.406309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:54.406309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:54.406309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:54.406309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:54.407462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:54.407462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:54.407462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:54.407462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:54.407462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:54.407462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:54.407462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:54.407462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:54.407462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:54.407462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:54.407462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:54.407462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:54.407462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:54.407462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:54.407462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:54.407462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:54.408271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:54.408271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:54.408271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:54.408271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:54.408271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:54.408271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:54.408271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:54.408271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:54.408271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:54.408271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:54.408271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:20:54.408271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:54.408271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 250/497] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:20:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:54.408937 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef547924-4b10-41ef-8cf7-695aed677ab7 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:54.469438 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0490bd9-5d92-48d0-85cf-6eee06a343db None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:54.504680 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b7837-5ef4-4fd4-9cea-2d68aef73a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:54.505343 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b7837-5ef4-4fd4-9cea-2d68aef73a29 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:54.505628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b7837-5ef4-4fd4-9cea-2d68aef73a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:54.505898 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b7837-5ef4-4fd4-9cea-2d68aef73a29 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:54.506686 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b7837-5ef4-4fd4-9cea-2d68aef73a29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:54.506929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b7837-5ef4-4fd4-9cea-2d68aef73a29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:54.508333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b7837-5ef4-4fd4-9cea-2d68aef73a29 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:54.516807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef547924-4b10-41ef-8cf7-695aed677ab7 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:54.520408 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 248/498] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:54.521625 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a669ee7-4068-489b-91b9-d1f0408324df None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:54.521914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a669ee7-4068-489b-91b9-d1f0408324df None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:54.522192 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a669ee7-4068-489b-91b9-d1f0408324df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:54.539598 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-7a669ee7-4068-489b-91b9-d1f0408324df None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:54.540809 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7a669ee7-4068-489b-91b9-d1f0408324df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:54.541946 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:54.552933 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10411676-9490-4102-a05d-f95a92f0f556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:54.553591 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10411676-9490-4102-a05d-f95a92f0f556 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:54.553876 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10411676-9490-4102-a05d-f95a92f0f556 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:54.554154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10411676-9490-4102-a05d-f95a92f0f556 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:54.565139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:54.611770 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 249/499] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:54.612641 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a669ee7-4068-489b-91b9-d1f0408324df None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:54.630696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10411676-9490-4102-a05d-f95a92f0f556 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:54.633737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10411676-9490-4102-a05d-f95a92f0f556 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:54.728138 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a669ee7-4068-489b-91b9-d1f0408324df None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:54.742150 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 251/500] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:54.743518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a1056-b88e-42b6-85dd-0e6cf5fe6e9d None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:54.743802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a1056-b88e-42b6-85dd-0e6cf5fe6e9d None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:54.744180 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a1056-b88e-42b6-85dd-0e6cf5fe6e9d None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:54.745498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 252/501] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:20:54.746009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a1056-b88e-42b6-85dd-0e6cf5fe6e9d None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:54.764606 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271701ca-ea5f-473f-957b-fe16d859a1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:54.765199 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271701ca-ea5f-473f-957b-fe16d859a1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:54.765480 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271701ca-ea5f-473f-957b-fe16d859a1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:54.765754 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271701ca-ea5f-473f-957b-fe16d859a1c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:54.766498 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271701ca-ea5f-473f-957b-fe16d859a1c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:54.766802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271701ca-ea5f-473f-957b-fe16d859a1c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:54.768301 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271701ca-ea5f-473f-957b-fe16d859a1c4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:54.771740 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:54.778399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 250/502] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:20:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:54.779883 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271701ca-ea5f-473f-957b-fe16d859a1c4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:54.804764 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:54.859497 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a1056-b88e-42b6-85dd-0e6cf5fe6e9d None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:54.881830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271701ca-ea5f-473f-957b-fe16d859a1c4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:54.896112 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca901786-6545-4a8d-8340-8fded5079310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:54.896913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca901786-6545-4a8d-8340-8fded5079310 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:54.897191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca901786-6545-4a8d-8340-8fded5079310 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:54.897459 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca901786-6545-4a8d-8340-8fded5079310 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:54.905025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca901786-6545-4a8d-8340-8fded5079310 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:54.906472 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca901786-6545-4a8d-8340-8fded5079310 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:54.917274 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc91e1-5693-40dd-b39f-89d28b499f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:54.918064 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc91e1-5693-40dd-b39f-89d28b499f86 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:54.918376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc91e1-5693-40dd-b39f-89d28b499f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:54.918652 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc91e1-5693-40dd-b39f-89d28b499f86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:54.923168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-ca901786-6545-4a8d-8340-8fded5079310 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 20:20:54.923168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:54.923168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:54.923168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:54.923168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:54.923168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:54.923168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:54.923168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:54.923168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:54.923168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:54.923168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:54.923168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:54.923168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:20:54.923168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:20:54.923168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:20:54.923168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:20:54.924129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:54.924129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:54.924129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:20:54.924129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:20:54.924129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:20:54.924129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:20:54.924129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:54.924129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:54.924129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:54.924129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:54.924129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:54.924129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:54.924129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:54.924129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:54.924129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:54.924129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:54.924129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:54.924129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:54.925013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:54.925013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:54.925013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:20:54.925013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:20:54.925013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:20:54.925013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:20:54.925013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:20:54.925013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:20:54.925013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 20:20:54.925013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:54.925013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 253/503] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:54.925557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca901786-6545-4a8d-8340-8fded5079310 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:54.931915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc91e1-5693-40dd-b39f-89d28b499f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:54.941304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-35bc91e1-5693-40dd-b39f-89d28b499f86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:20:54.941304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:54.941304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:54.941304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:54.941304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:54.941304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:54.941304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:54.941304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:54.941304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:54.941304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:54.941304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:54.941304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:54.941304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:54.941304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:54.941304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:54.941304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:54.942288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:54.942288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:54.942288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:54.942288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:54.942288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:54.942288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:54.942288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:54.942288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:54.942288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:54.942288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:54.942288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:54.942288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:54.942288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:54.942288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:54.942288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:54.942288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:54.943129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:54.943129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:54.943129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:54.943129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:54.943129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:54.943129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:54.943129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:54.943129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:54.943129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:54.943129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:54.943129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:20:54.943129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:54.943129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 251/504] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:20:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:54.943825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a33c4a-1fe0-4b4e-80bf-4e4ea8c24311 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:54.944105 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a33c4a-1fe0-4b4e-80bf-4e4ea8c24311 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:54.944380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a33c4a-1fe0-4b4e-80bf-4e4ea8c24311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:54.961851 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c2a33c4a-1fe0-4b4e-80bf-4e4ea8c24311 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:54.963089 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c2a33c4a-1fe0-4b4e-80bf-4e4ea8c24311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:55.015831 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:55.032403 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 252/505] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:55.033273 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a33c4a-1fe0-4b4e-80bf-4e4ea8c24311 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:55.033377 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca901786-6545-4a8d-8340-8fded5079310 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:55.056301 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:55.071683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fc9fc-3d95-47a7-963e-5f6ddd68decc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:55.072592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fc9fc-3d95-47a7-963e-5f6ddd68decc None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:55.072759 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fc9fc-3d95-47a7-963e-5f6ddd68decc None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:55.073028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fc9fc-3d95-47a7-963e-5f6ddd68decc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:55.083789 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066fc9fc-3d95-47a7-963e-5f6ddd68decc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:55.085250 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066fc9fc-3d95-47a7-963e-5f6ddd68decc None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:55.104131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-066fc9fc-3d95-47a7-963e-5f6ddd68decc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:55.104131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:55.104131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:55.104131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:55.104131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:55.104131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:55.104131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:55.104131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:55.104131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:55.104131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:55.104131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:55.104131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:55.104131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:20:55.104131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:20:55.104131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:20:55.105275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:20:55.105275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:55.105275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:55.105275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:55.105275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:55.105275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:55.105275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:55.105275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:55.105275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:55.105275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:55.105275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:55.105275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:55.105275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:55.105275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:55.105275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:55.105275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:55.105275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:55.106340 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:20:55.106340 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:20:55.106340 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:55.106340 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:55.106340 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 254/506] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:55.106340 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fc9fc-3d95-47a7-963e-5f6ddd68decc None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:55.152088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a33c4a-1fe0-4b4e-80bf-4e4ea8c24311 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:55.187057 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19153f76-917c-4e00-8f50-9e597911a912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:55.187872 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19153f76-917c-4e00-8f50-9e597911a912 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:55.188152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19153f76-917c-4e00-8f50-9e597911a912 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:55.188428 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19153f76-917c-4e00-8f50-9e597911a912 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:55.199549 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19153f76-917c-4e00-8f50-9e597911a912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:55.201028 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19153f76-917c-4e00-8f50-9e597911a912 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:55.207273 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fc9fc-3d95-47a7-963e-5f6ddd68decc None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:55.218335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-19153f76-917c-4e00-8f50-9e597911a912 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:55.218335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:55.218335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:55.218335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:55.218335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:55.218335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:55.218335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:55.218335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:55.218335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:55.218335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:55.218335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:55.218335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:55.218335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:20:55.218335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:20:55.218335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:20:55.219371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:20:55.219371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:55.219371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:55.219371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:55.219371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:55.219371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:55.219371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:55.219371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:55.219371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:55.219371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:55.219371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:55.219371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:55.219371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:55.219371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:55.219371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:55.219371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:55.219371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:55.220243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:20:55.220243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:20:55.220243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:55.220243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:55.220243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 253/507] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:55.220535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19153f76-917c-4e00-8f50-9e597911a912 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:55.245733 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089c9f4-9f99-480e-8bd2-b30301f46124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:55.246411 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089c9f4-9f99-480e-8bd2-b30301f46124 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:55.246698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089c9f4-9f99-480e-8bd2-b30301f46124 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:55.247010 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089c9f4-9f99-480e-8bd2-b30301f46124 None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:55.248409 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089c9f4-9f99-480e-8bd2-b30301f46124 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:55.248712 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089c9f4-9f99-480e-8bd2-b30301f46124 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:55.250155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089c9f4-9f99-480e-8bd2-b30301f46124 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:55.261487 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:55.282806 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:55.288315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 255/508] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jul 14 20:20:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:55.289182 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089c9f4-9f99-480e-8bd2-b30301f46124 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:55.323172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19153f76-917c-4e00-8f50-9e597911a912 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:55.357837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34caae-0091-499a-842f-b3696cf3b9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:55.358498 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34caae-0091-499a-842f-b3696cf3b9ac None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:55.358781 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34caae-0091-499a-842f-b3696cf3b9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:55.359093 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34caae-0091-499a-842f-b3696cf3b9ac None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:55.359846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34caae-0091-499a-842f-b3696cf3b9ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:55.360146 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34caae-0091-499a-842f-b3696cf3b9ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:55.361548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34caae-0091-499a-842f-b3696cf3b9ac None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:55.369919 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 254/509] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:20:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:55.370555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34caae-0091-499a-842f-b3696cf3b9ac None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:55.395693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089c9f4-9f99-480e-8bd2-b30301f46124 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:55.430109 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a7846-4495-4906-aa5e-6d8e5ec3e206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:55.430902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a7846-4495-4906-aa5e-6d8e5ec3e206 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:55.431214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a7846-4495-4906-aa5e-6d8e5ec3e206 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:55.431487 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a7846-4495-4906-aa5e-6d8e5ec3e206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:55.441962 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a7846-4495-4906-aa5e-6d8e5ec3e206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:55.443618 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a7846-4495-4906-aa5e-6d8e5ec3e206 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:55.461072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-300a7846-4495-4906-aa5e-6d8e5ec3e206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:55.461072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:55.461072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:55.461072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:55.461072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:55.461072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:55.461072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:55.461072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:55.461072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:55.461072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:55.461072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:55.461072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:55.461072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:20:55.461072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:20:55.461072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:20:55.462011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:20:55.462011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:55.462011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:55.462011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:55.462011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:55.462011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:55.462011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:55.462011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:55.462011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:55.462011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:55.462011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:55.462011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:55.462011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:55.462011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:55.462011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:55.462011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:55.462011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:55.462850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:20:55.462850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:20:55.462850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:55.462850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:55.462850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 256/510] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:55.467941 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a7846-4495-4906-aa5e-6d8e5ec3e206 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:55.483015 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34caae-0091-499a-842f-b3696cf3b9ac None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:55.512441 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:55.520106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:55.523126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25c6c9-39a0-4550-96fa-3ea50f934747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:55.523798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25c6c9-39a0-4550-96fa-3ea50f934747 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:55.524085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25c6c9-39a0-4550-96fa-3ea50f934747 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:55.524359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25c6c9-39a0-4550-96fa-3ea50f934747 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:55.525094 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25c6c9-39a0-4550-96fa-3ea50f934747 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:55.525397 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25c6c9-39a0-4550-96fa-3ea50f934747 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:55.526790 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25c6c9-39a0-4550-96fa-3ea50f934747 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:55.539472 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 255/511] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:55.540252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25c6c9-39a0-4550-96fa-3ea50f934747 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:55.578996 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a7846-4495-4906-aa5e-6d8e5ec3e206 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:55.613789 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09aa1a-2cad-4d9e-979b-654896abf48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:55.614456 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09aa1a-2cad-4d9e-979b-654896abf48e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:55.614735 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09aa1a-2cad-4d9e-979b-654896abf48e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:55.615041 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09aa1a-2cad-4d9e-979b-654896abf48e None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:55.615769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09aa1a-2cad-4d9e-979b-654896abf48e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:55.616074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09aa1a-2cad-4d9e-979b-654896abf48e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:55.617460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09aa1a-2cad-4d9e-979b-654896abf48e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:55.632348 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 257/512] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:55.633208 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09aa1a-2cad-4d9e-979b-654896abf48e None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:55.656272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25c6c9-39a0-4550-96fa-3ea50f934747 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:55.690777 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca4fad-4660-4c05-8959-e692b14d7124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:55.691455 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca4fad-4660-4c05-8959-e692b14d7124 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:55.691738 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca4fad-4660-4c05-8959-e692b14d7124 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:55.692013 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca4fad-4660-4c05-8959-e692b14d7124 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:55.739419 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09aa1a-2cad-4d9e-979b-654896abf48e None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:55.747406 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:55.747551 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:55.761289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca4fad-4660-4c05-8959-e692b14d7124 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:55.762774 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca4fad-4660-4c05-8959-e692b14d7124 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:55.775925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df64e2-6ee5-46a9-b672-d4d5b01275b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:55.776722 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df64e2-6ee5-46a9-b672-d4d5b01275b4 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:55.777000 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df64e2-6ee5-46a9-b672-d4d5b01275b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:55.777274 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df64e2-6ee5-46a9-b672-d4d5b01275b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:55.788060 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df64e2-6ee5-46a9-b672-d4d5b01275b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:55.789601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df64e2-6ee5-46a9-b672-d4d5b01275b4 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:55.807862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-53df64e2-6ee5-46a9-b672-d4d5b01275b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:55.807862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:55.807862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:55.807862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:55.807862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:55.807862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:55.807862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:55.807862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:55.807862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:55.807862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:55.807862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:55.807862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:55.807862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:20:55.807862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:20:55.807862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:20:55.808775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:20:55.808775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:55.808775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:55.808775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:55.808775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:55.808775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:55.808775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:55.808775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:55.808775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:55.808775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:55.808775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:55.808775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:55.808775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:55.808775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:55.808775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:55.808775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:55.808775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:55.809848 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:20:55.809848 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:20:55.809848 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:55.809848 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:55.809848 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 258/513] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:55.810284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df64e2-6ee5-46a9-b672-d4d5b01275b4 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:55.868735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 256/514] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:55.869584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca4fad-4660-4c05-8959-e692b14d7124 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:55.910237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df64e2-6ee5-46a9-b672-d4d5b01275b4 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:55.945633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ee936-06de-4b6c-be15-3326e737fdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:55.946310 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ee936-06de-4b6c-be15-3326e737fdba None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:55.946590 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ee936-06de-4b6c-be15-3326e737fdba None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:55.946871 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ee936-06de-4b6c-be15-3326e737fdba None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:55.947637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ee936-06de-4b6c-be15-3326e737fdba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:55.947946 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ee936-06de-4b6c-be15-3326e737fdba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:55.949319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ee936-06de-4b6c-be15-3326e737fdba None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:55.960649 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:55.965858 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 259/515] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:55.965858 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ee936-06de-4b6c-be15-3326e737fdba None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:55.978553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca4fad-4660-4c05-8959-e692b14d7124 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:55.981452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:56.013237 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b823b64-daa9-40a7-9a84-dcaacd4158cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:56.014574 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b823b64-daa9-40a7-9a84-dcaacd4158cc None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:56.014860 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b823b64-daa9-40a7-9a84-dcaacd4158cc None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:56.015167 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b823b64-daa9-40a7-9a84-dcaacd4158cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:56.022644 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b823b64-daa9-40a7-9a84-dcaacd4158cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:56.024123 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b823b64-daa9-40a7-9a84-dcaacd4158cc None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:56.038969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-6b823b64-daa9-40a7-9a84-dcaacd4158cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:20:56.038969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:56.038969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:56.038969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:56.038969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:56.038969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:56.038969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:56.038969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:56.038969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:56.038969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:56.038969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:56.038969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:56.038969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:20:56.038969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:20:56.038969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:20:56.040118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:20:56.040118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:56.040118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:56.040118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:56.040118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:56.040118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:56.040118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:56.040118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:56.040118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:56.040118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:56.040118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:56.040118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:56.040118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:56.040118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:56.040118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:56.040118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:56.040118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:56.041406 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:20:56.041406 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:20:56.041406 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:20:56.041406 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:20:56.041406 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:20:56.041406 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:20:56.041406 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:20:56.041406 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:56.041406 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 257/516] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:20:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:56.041406 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b823b64-daa9-40a7-9a84-dcaacd4158cc None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:56.068122 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ee936-06de-4b6c-be15-3326e737fdba None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:56.108594 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408a3e0-5690-4f50-9822-237acd91c637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:56.108594 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408a3e0-5690-4f50-9822-237acd91c637 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:56.108864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408a3e0-5690-4f50-9822-237acd91c637 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:56.109132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408a3e0-5690-4f50-9822-237acd91c637 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:56.116583 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408a3e0-5690-4f50-9822-237acd91c637 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:56.118025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408a3e0-5690-4f50-9822-237acd91c637 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:56.132229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-c408a3e0-5690-4f50-9822-237acd91c637 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 20:20:56.132229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:56.132229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:56.132229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:56.132229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:56.132229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:56.132229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:56.132229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:56.132229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:56.132229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:56.132229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:56.132229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:56.132229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:20:56.132229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:20:56.132229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:20:56.132229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:20:56.133288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:56.133288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:56.133288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:20:56.133288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:20:56.133288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:20:56.133288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:20:56.133288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:56.133288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:56.133288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:56.133288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:56.133288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:56.133288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:56.133288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:56.133288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:56.133288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:56.133288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:56.133288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:56.133288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:56.134304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:56.134304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:56.134304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:20:56.134304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:20:56.134304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:20:56.134304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:20:56.134304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:20:56.134304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:20:56.134304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 20:20:56.134304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:56.134304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 260/517] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:20:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:56.134304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408a3e0-5690-4f50-9822-237acd91c637 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:56.144507 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b823b64-daa9-40a7-9a84-dcaacd4158cc None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:56.179935 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65473d6a-9f51-413c-9a01-73235d3b3308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:56.180543 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65473d6a-9f51-413c-9a01-73235d3b3308 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:56.180830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65473d6a-9f51-413c-9a01-73235d3b3308 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:56.181107 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65473d6a-9f51-413c-9a01-73235d3b3308 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:56.181846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65473d6a-9f51-413c-9a01-73235d3b3308 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:56.182153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65473d6a-9f51-413c-9a01-73235d3b3308 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:56.183564 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65473d6a-9f51-413c-9a01-73235d3b3308 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:56.196395 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 258/518] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:56.197229 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65473d6a-9f51-413c-9a01-73235d3b3308 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:56.207445 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:56.227474 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:56.249740 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408a3e0-5690-4f50-9822-237acd91c637 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:56.284393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1072a-a395-471e-a978-083d7fbb0c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:56.285314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1072a-a395-471e-a978-083d7fbb0c84 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:56.285624 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1072a-a395-471e-a978-083d7fbb0c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:56.285939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1072a-a395-471e-a978-083d7fbb0c84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:56.298594 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1072a-a395-471e-a978-083d7fbb0c84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:56.300022 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1072a-a395-471e-a978-083d7fbb0c84 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:56.303915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65473d6a-9f51-413c-9a01-73235d3b3308 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:56.318003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-8af1072a-a395-471e-a978-083d7fbb0c84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:56.318003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:56.318003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:56.318003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:56.318003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:56.318003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:56.318003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:56.318003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:56.318003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:56.318003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:56.318003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:56.318003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:56.318003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:20:56.318003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:20:56.318003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:20:56.319114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:20:56.319114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:56.319114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:56.319114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:56.319114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:56.319114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:56.319114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:56.319114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:56.319114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:56.319114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:56.319114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:56.319114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:56.319114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:56.319114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:56.319114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:56.319114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:56.319114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:56.324726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:20:56.324726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:20:56.324726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:56.324726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:56.324726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 261/519] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:56.324726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1072a-a395-471e-a978-083d7fbb0c84 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:56.348790 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876010c-d75d-4e0a-921b-93c3543fb5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:56.349476 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876010c-d75d-4e0a-921b-93c3543fb5cb None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:56.349761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876010c-d75d-4e0a-921b-93c3543fb5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:56.350037 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876010c-d75d-4e0a-921b-93c3543fb5cb None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:56.350743 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876010c-d75d-4e0a-921b-93c3543fb5cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:56.352226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876010c-d75d-4e0a-921b-93c3543fb5cb None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:56.373500 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 259/520] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 14 20:20:56 2022] POST /identity/v3/users => generated 247 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:20:56.374144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876010c-d75d-4e0a-921b-93c3543fb5cb None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:56.436073 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1072a-a395-471e-a978-083d7fbb0c84 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:56.449232 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:56.469263 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:56.477067 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c8ab9-7e9d-4539-b26f-26fa348ec984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:56.477758 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c8ab9-7e9d-4539-b26f-26fa348ec984 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:56.478045 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c8ab9-7e9d-4539-b26f-26fa348ec984 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:56.478324 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c8ab9-7e9d-4539-b26f-26fa348ec984 None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:56.479178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c8ab9-7e9d-4539-b26f-26fa348ec984 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:56.479490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c8ab9-7e9d-4539-b26f-26fa348ec984 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:56.480992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c8ab9-7e9d-4539-b26f-26fa348ec984 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:56.501382 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 262/521] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:20:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:56.502886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c8ab9-7e9d-4539-b26f-26fa348ec984 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:56.508784 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876010c-d75d-4e0a-921b-93c3543fb5cb None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:56.552686 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1a2b2-48e8-4b42-a35e-97af4555f681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:56.553289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1a2b2-48e8-4b42-a35e-97af4555f681 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:56.553573 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1a2b2-48e8-4b42-a35e-97af4555f681 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:56.553848 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1a2b2-48e8-4b42-a35e-97af4555f681 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:56.554691 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1a2b2-48e8-4b42-a35e-97af4555f681 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:56.555073 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1a2b2-48e8-4b42-a35e-97af4555f681 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:56.556518 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1a2b2-48e8-4b42-a35e-97af4555f681 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:56.578171 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 260/522] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jul 14 20:20:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:56.579132 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1a2b2-48e8-4b42-a35e-97af4555f681 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:56.629204 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c8ab9-7e9d-4539-b26f-26fa348ec984 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:56.665403 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22499814-7002-4a67-a78a-a53db8cd3884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:56.666218 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22499814-7002-4a67-a78a-a53db8cd3884 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:56.666506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22499814-7002-4a67-a78a-a53db8cd3884 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:56.666786 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22499814-7002-4a67-a78a-a53db8cd3884 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:56.675249 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22499814-7002-4a67-a78a-a53db8cd3884 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:56.676720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22499814-7002-4a67-a78a-a53db8cd3884 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:56.692984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-22499814-7002-4a67-a78a-a53db8cd3884 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 20:20:56.692984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:56.692984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:56.692984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:56.692984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:56.692984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:56.692984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:56.692984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:56.692984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:56.692984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:56.692984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:56.692984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:56.692984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:20:56.692984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:20:56.692984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:20:56.692984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:20:56.694085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:56.694085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:56.694085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:20:56.694085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:20:56.694085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:20:56.694085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:20:56.694085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:56.694085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:56.694085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:56.694085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:56.694085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:56.694085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:56.694085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:56.694085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:56.694085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:56.694085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:56.694085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:56.694085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:56.695330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:56.695330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:56.695330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:20:56.695330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:20:56.695330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:20:56.695330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:20:56.695330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:20:56.695330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:20:56.695330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 20:20:56.695330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:56.695330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 263/523] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:20:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:56.695330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22499814-7002-4a67-a78a-a53db8cd3884 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:56.710843 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1a2b2-48e8-4b42-a35e-97af4555f681 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:56.733718 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:56.750196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea795c3-3843-4f1c-a793-921af5b5eb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:56.750433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:56.750870 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea795c3-3843-4f1c-a793-921af5b5eb4b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:56.751199 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea795c3-3843-4f1c-a793-921af5b5eb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:56.751479 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea795c3-3843-4f1c-a793-921af5b5eb4b None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:56.752374 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea795c3-3843-4f1c-a793-921af5b5eb4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:56.752686 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea795c3-3843-4f1c-a793-921af5b5eb4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:56.754388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea795c3-3843-4f1c-a793-921af5b5eb4b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:56.771242 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 261/524] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:56.771242 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea795c3-3843-4f1c-a793-921af5b5eb4b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:56.818383 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22499814-7002-4a67-a78a-a53db8cd3884 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:56.852697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17486ab-bda1-4450-88a3-bdb64dc43bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:56.853353 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17486ab-bda1-4450-88a3-bdb64dc43bff None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:56.853635 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17486ab-bda1-4450-88a3-bdb64dc43bff None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:56.853909 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17486ab-bda1-4450-88a3-bdb64dc43bff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:56.854745 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17486ab-bda1-4450-88a3-bdb64dc43bff None admin] RBAC: Adding query filter params (role.id=ea49c36b4a7f4cfb9048065638d458af, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=aa3bb9f1e24a4e9ebd460e97b3736bc0) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:56.855149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17486ab-bda1-4450-88a3-bdb64dc43bff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:56.856792 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17486ab-bda1-4450-88a3-bdb64dc43bff None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:56.863040 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 264/525] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:56 2022] GET /identity/v3/role_assignments?user.id=aa3bb9f1e24a4e9ebd460e97b3736bc0&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=ea49c36b4a7f4cfb9048065638d458af => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:56.863857 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17486ab-bda1-4450-88a3-bdb64dc43bff None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:56.918454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea795c3-3843-4f1c-a793-921af5b5eb4b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:56.961983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f5726-26a8-4782-bdd1-6a37533c620f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:56.962707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f5726-26a8-4782-bdd1-6a37533c620f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:56.963026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f5726-26a8-4782-bdd1-6a37533c620f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:56.963306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f5726-26a8-4782-bdd1-6a37533c620f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:56.995806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17486ab-bda1-4450-88a3-bdb64dc43bff None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:57.015667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:57.032343 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:57.042346 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc9ab3-3f0c-48af-8948-38fa55a98427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:57.043056 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc9ab3-3f0c-48af-8948-38fa55a98427 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:57.043351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc9ab3-3f0c-48af-8948-38fa55a98427 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:57.043623 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc9ab3-3f0c-48af-8948-38fa55a98427 None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:57.045299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc9ab3-3f0c-48af-8948-38fa55a98427 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:57.045299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc9ab3-3f0c-48af-8948-38fa55a98427 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:57.046990 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc9ab3-3f0c-48af-8948-38fa55a98427 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:57.061667 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f5726-26a8-4782-bdd1-6a37533c620f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:57.063685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f5726-26a8-4782-bdd1-6a37533c620f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:57.066988 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 265/526] 10.222.0.52 () {56 vars in 1139 bytes} [Thu Jul 14 20:20:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:57.068090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc9ab3-3f0c-48af-8948-38fa55a98427 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:57.193966 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 262/527] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:57.194564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc9ab3-3f0c-48af-8948-38fa55a98427 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:57.195333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f5726-26a8-4782-bdd1-6a37533c620f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:57.233865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84fcea-14a6-4e53-a1d3-29d10c92cba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:57.234664 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84fcea-14a6-4e53-a1d3-29d10c92cba0 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:57.234960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84fcea-14a6-4e53-a1d3-29d10c92cba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:57.235318 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84fcea-14a6-4e53-a1d3-29d10c92cba0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:57.243106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84fcea-14a6-4e53-a1d3-29d10c92cba0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:57.244580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84fcea-14a6-4e53-a1d3-29d10c92cba0 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:57.259856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-1f84fcea-14a6-4e53-a1d3-29d10c92cba0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:20:57.259856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:57.259856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:57.259856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:57.259856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:57.259856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:57.259856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:57.259856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:57.259856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:57.259856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:57.259856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:57.259856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:57.259856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:20:57.259856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:20:57.259856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:20:57.260763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:20:57.260763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:57.260763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:57.260763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:57.260763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:57.260763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:57.260763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:57.260763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:57.260763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:57.260763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:57.260763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:57.260763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:57.260763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:57.260763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:57.260763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:57.260763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:57.260763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:57.261683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:20:57.261683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:20:57.261683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:20:57.261683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:20:57.261683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:20:57.261683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:20:57.261683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:20:57.261683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:57.265786 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 266/528] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:20:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:57.266731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84fcea-14a6-4e53-a1d3-29d10c92cba0 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:57.300894 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:57.308470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:57.326122 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f5726-26a8-4782-bdd1-6a37533c620f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:57.365740 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a4ad0-331e-4676-a895-8e66d1f08eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:57.366429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a4ad0-331e-4676-a895-8e66d1f08eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:57.366714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a4ad0-331e-4676-a895-8e66d1f08eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:57.367015 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a4ad0-331e-4676-a895-8e66d1f08eb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:57.396567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84fcea-14a6-4e53-a1d3-29d10c92cba0 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:57.431726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4859f-f204-4889-a030-9271bdbdd988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:57.432469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4859f-f204-4889-a030-9271bdbdd988 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:57.432753 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4859f-f204-4889-a030-9271bdbdd988 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:57.433027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4859f-f204-4889-a030-9271bdbdd988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:57.447075 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4859f-f204-4889-a030-9271bdbdd988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:57.448531 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4859f-f204-4889-a030-9271bdbdd988 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:57.449484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a4ad0-331e-4676-a895-8e66d1f08eb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:57.452939 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a4ad0-331e-4676-a895-8e66d1f08eb2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:57.467925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-c3c4859f-f204-4889-a030-9271bdbdd988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:57.467925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:57.467925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:57.467925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:57.467925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:57.467925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:57.467925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:57.467925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:57.467925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:57.467925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:57.467925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:57.467925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:57.467925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:20:57.467925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:20:57.467925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:20:57.468836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:20:57.468836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:57.468836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:57.468836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:57.468836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:57.468836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:57.468836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:57.468836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:57.468836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:57.468836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:57.468836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:57.468836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:57.468836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:57.468836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:57.468836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:57.468836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:57.468836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:57.469684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:20:57.469684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:20:57.469684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:57.469684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:57.474198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 267/529] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:57.475259 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4859f-f204-4889-a030-9271bdbdd988 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:57.564299 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:57.580296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4859f-f204-4889-a030-9271bdbdd988 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:57.583281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:57.601348 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 263/530] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:20:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:20:57.602408 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a4ad0-331e-4676-a895-8e66d1f08eb2 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:57.624404 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c2b32-53af-4999-a546-6b02f32290be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:57.625137 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c2b32-53af-4999-a546-6b02f32290be None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:57.625493 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c2b32-53af-4999-a546-6b02f32290be None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:57.625868 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c2b32-53af-4999-a546-6b02f32290be None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:57.626758 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c2b32-53af-4999-a546-6b02f32290be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:57.627172 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c2b32-53af-4999-a546-6b02f32290be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:57.629160 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c2b32-53af-4999-a546-6b02f32290be None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:57.643559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 268/531] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:20:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:57.644891 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c2b32-53af-4999-a546-6b02f32290be None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:57.718903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a4ad0-331e-4676-a895-8e66d1f08eb2 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:57.756523 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d9693-2a3e-4ad8-afb1-55a915a23963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:57.757609 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d9693-2a3e-4ad8-afb1-55a915a23963 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:57.758092 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d9693-2a3e-4ad8-afb1-55a915a23963 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:57.758565 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d9693-2a3e-4ad8-afb1-55a915a23963 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:57.758899 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c2b32-53af-4999-a546-6b02f32290be None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:57.760077 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d9693-2a3e-4ad8-afb1-55a915a23963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:57.760637 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d9693-2a3e-4ad8-afb1-55a915a23963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:57.762328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d9693-2a3e-4ad8-afb1-55a915a23963 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:57.775930 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 264/532] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:57.777233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d9693-2a3e-4ad8-afb1-55a915a23963 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:57.796524 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc95c7-dfa9-478e-8520-a906ff6e732c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:57.797343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc95c7-dfa9-478e-8520-a906ff6e732c None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:57.797631 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc95c7-dfa9-478e-8520-a906ff6e732c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:57.797911 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc95c7-dfa9-478e-8520-a906ff6e732c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:57.809081 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc95c7-dfa9-478e-8520-a906ff6e732c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:57.811326 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc95c7-dfa9-478e-8520-a906ff6e732c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:57.815014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:57.828478 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:57.839458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-dbdc95c7-dfa9-478e-8520-a906ff6e732c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:57.839458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:57.839458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:57.839458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:57.839458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:57.839458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:57.839458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:57.839458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:57.839458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:57.839458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:57.839458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:57.839458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:57.839458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:20:57.839458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:20:57.839458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:20:57.840349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:20:57.840349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:57.840349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:57.840349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:57.840349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:57.840349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:57.840349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:57.840349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:57.840349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:57.840349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:57.840349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:57.840349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:57.840349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:57.840349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:57.840349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:57.840349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:57.840349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:57.841200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:20:57.841200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:20:57.841200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:57.841200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:57.845939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 269/533] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:57.847093 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc95c7-dfa9-478e-8520-a906ff6e732c None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:57.908062 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d9693-2a3e-4ad8-afb1-55a915a23963 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:57.947600 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e36d5c-c512-4275-b728-521773d6185d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:57.948856 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e36d5c-c512-4275-b728-521773d6185d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:57.948856 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e36d5c-c512-4275-b728-521773d6185d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:57.949027 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e36d5c-c512-4275-b728-521773d6185d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:57.949765 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e36d5c-c512-4275-b728-521773d6185d None admin] RBAC: Adding query filter params (role.id=ea49c36b4a7f4cfb9048065638d458af, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=4b7a38b4a7754037a00d11dafc88645b) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:57.950080 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e36d5c-c512-4275-b728-521773d6185d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:57.952204 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e36d5c-c512-4275-b728-521773d6185d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:57.958857 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 265/534] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:57 2022] GET /identity/v3/role_assignments?user.id=4b7a38b4a7754037a00d11dafc88645b&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=ea49c36b4a7f4cfb9048065638d458af => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:57.960019 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e36d5c-c512-4275-b728-521773d6185d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:57.991284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc95c7-dfa9-478e-8520-a906ff6e732c None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:58.046986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601025c2-3d72-4248-9818-87600ad5f37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:58.046986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601025c2-3d72-4248-9818-87600ad5f37b None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:58.046986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601025c2-3d72-4248-9818-87600ad5f37b None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:58.046986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601025c2-3d72-4248-9818-87600ad5f37b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:58.063376 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601025c2-3d72-4248-9818-87600ad5f37b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:58.065133 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601025c2-3d72-4248-9818-87600ad5f37b None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:58.082992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-601025c2-3d72-4248-9818-87600ad5f37b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:20:58.082992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:58.082992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:58.082992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:58.082992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:58.082992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:58.082992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:58.082992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:58.082992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:58.082992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:58.082992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:58.082992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:58.082992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:20:58.082992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:20:58.082992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:20:58.084058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:20:58.084058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:58.084058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:58.084058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:58.084058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:58.084058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:58.084058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:58.084058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:58.084058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:58.084058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:58.084058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:58.084058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:58.084058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:58.084058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:58.084058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:58.084058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:58.084058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:58.085078 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:20:58.085078 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:20:58.085078 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:20:58.085078 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:20:58.085078 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:20:58.085078 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:20:58.085078 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:20:58.085078 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:58.088317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 270/535] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:20:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:58.094347 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601025c2-3d72-4248-9818-87600ad5f37b None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:58.150347 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e36d5c-c512-4275-b728-521773d6185d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:58.167111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:58.200709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f8f41-6fac-4531-adbd-a2b705d50d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:58.202843 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f8f41-6fac-4531-adbd-a2b705d50d73 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:58.203619 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f8f41-6fac-4531-adbd-a2b705d50d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:58.204278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f8f41-6fac-4531-adbd-a2b705d50d73 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:58.205239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f8f41-6fac-4531-adbd-a2b705d50d73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:58.205983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f8f41-6fac-4531-adbd-a2b705d50d73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:58.209027 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f8f41-6fac-4531-adbd-a2b705d50d73 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:58.216642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:58.231645 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 266/536] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:58.238667 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f8f41-6fac-4531-adbd-a2b705d50d73 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:58.337173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601025c2-3d72-4248-9818-87600ad5f37b None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:58.408060 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5d157-5b33-484b-8579-89e831256bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:58.410091 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5d157-5b33-484b-8579-89e831256bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:58.410782 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5d157-5b33-484b-8579-89e831256bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:58.411920 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5d157-5b33-484b-8579-89e831256bc7 None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:58.413741 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5d157-5b33-484b-8579-89e831256bc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:58.414431 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5d157-5b33-484b-8579-89e831256bc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:58.416610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f8f41-6fac-4531-adbd-a2b705d50d73 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:58.418316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5d157-5b33-484b-8579-89e831256bc7 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:58.434434 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 271/537] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:20:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:58.457701 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5d157-5b33-484b-8579-89e831256bc7 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:58.485374 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba0404-bf5f-4244-9327-dfe2661104e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:58.485986 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba0404-bf5f-4244-9327-dfe2661104e3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:58.486274 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba0404-bf5f-4244-9327-dfe2661104e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:58.486549 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba0404-bf5f-4244-9327-dfe2661104e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:58.497135 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba0404-bf5f-4244-9327-dfe2661104e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:58.497135 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba0404-bf5f-4244-9327-dfe2661104e3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:58.529830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-6dba0404-bf5f-4244-9327-dfe2661104e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:20:58.529830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:58.529830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:58.529830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:58.529830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:58.529830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:58.529830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:58.529830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:58.529830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:58.529830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:58.529830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:58.529830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:58.529830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:20:58.529830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:20:58.529830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:20:58.530755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:20:58.530755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:58.530755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:58.530755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:58.530755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:58.530755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:58.530755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:58.530755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:58.530755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:58.530755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:58.530755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:58.530755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:58.530755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:58.530755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:58.530755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:58.530755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:58.530755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:58.531638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:20:58.531638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:20:58.531638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:20:58.531638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:20:58.531638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:20:58.531638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:20:58.531638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:20:58.531638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:58.531638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 267/538] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:20:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:58.541297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba0404-bf5f-4244-9327-dfe2661104e3 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:58.604180 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:58.652552 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5d157-5b33-484b-8579-89e831256bc7 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:58.697205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba0404-bf5f-4244-9327-dfe2661104e3 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:58.701937 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:58.737492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe36c5-8584-44c0-b6c0-9a792a0c6936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:58.738164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe36c5-8584-44c0-b6c0-9a792a0c6936 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:58.738453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe36c5-8584-44c0-b6c0-9a792a0c6936 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:58.738732 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe36c5-8584-44c0-b6c0-9a792a0c6936 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:58.739638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe36c5-8584-44c0-b6c0-9a792a0c6936 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:58.739954 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe36c5-8584-44c0-b6c0-9a792a0c6936 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:58.740206 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb92c29-01d1-41b4-bc5b-c025e55974e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:58.740880 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb92c29-01d1-41b4-bc5b-c025e55974e2 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:58.741167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb92c29-01d1-41b4-bc5b-c025e55974e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:58.741403 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe36c5-8584-44c0-b6c0-9a792a0c6936 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:58.741499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb92c29-01d1-41b4-bc5b-c025e55974e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:58.742274 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb92c29-01d1-41b4-bc5b-c025e55974e2 None admin] RBAC: Adding query filter params (role.id=ea49c36b4a7f4cfb9048065638d458af, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=01b4142ea6f142df9b1314532c961f75) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:58.742591 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb92c29-01d1-41b4-bc5b-c025e55974e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:58.744068 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb92c29-01d1-41b4-bc5b-c025e55974e2 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:58.758707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 272/539] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:58 2022] GET /identity/v3/role_assignments?user.id=01b4142ea6f142df9b1314532c961f75&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=ea49c36b4a7f4cfb9048065638d458af => generated 259 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:58.760339 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 268/540] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:20:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:20:58.766178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb92c29-01d1-41b4-bc5b-c025e55974e2 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:58.826033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ffa81-ecd0-46aa-834e-e53130c869c6 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:58.826339 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ffa81-ecd0-46aa-834e-e53130c869c6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:58.826716 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ffa81-ecd0-46aa-834e-e53130c869c6 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:58.828547 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 269/541] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:20:58.834377 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06e3b9-8bc9-46ed-bc4f-21ed567cd90b None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:58.834665 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06e3b9-8bc9-46ed-bc4f-21ed567cd90b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:58.834952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06e3b9-8bc9-46ed-bc4f-21ed567cd90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:58.854817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-da06e3b9-8bc9-46ed-bc4f-21ed567cd90b None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:58.856013 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-da06e3b9-8bc9-46ed-bc4f-21ed567cd90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:58.941589 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 270/542] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:20:58.959104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb92c29-01d1-41b4-bc5b-c025e55974e2 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:58.967593 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06e3b9-8bc9-46ed-bc4f-21ed567cd90b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:58.992516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:59.025073 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e240b9-d727-408f-86ac-5e35852adf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:59.025748 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e240b9-d727-408f-86ac-5e35852adf2f None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:59.026032 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e240b9-d727-408f-86ac-5e35852adf2f None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:59.026307 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e240b9-d727-408f-86ac-5e35852adf2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:59.027183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e240b9-d727-408f-86ac-5e35852adf2f None admin] RBAC: Adding query filter params (role.id=ea49c36b4a7f4cfb9048065638d458af, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=f88257850ee34ad895208ab935b6e643) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:59.027501 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e240b9-d727-408f-86ac-5e35852adf2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:59.029003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e240b9-d727-408f-86ac-5e35852adf2f None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:59.043805 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 273/543] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:20:58 2022] GET /identity/v3/role_assignments?user.id=f88257850ee34ad895208ab935b6e643&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=ea49c36b4a7f4cfb9048065638d458af => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:59.062415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889e07a-86ac-49a8-9fbc-eefcac19f967 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:59.062752 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889e07a-86ac-49a8-9fbc-eefcac19f967 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:59.063013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889e07a-86ac-49a8-9fbc-eefcac19f967 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:59.064626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 274/544] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:20:59.072696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3688355-452f-474f-8e48-d2f5738b96eb None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:59.073149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3688355-452f-474f-8e48-d2f5738b96eb None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:59.074047 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3688355-452f-474f-8e48-d2f5738b96eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:59.107599 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-f3688355-452f-474f-8e48-d2f5738b96eb None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:20:59.109220 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f3688355-452f-474f-8e48-d2f5738b96eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:20:59.152446 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06e3b9-8bc9-46ed-bc4f-21ed567cd90b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:59.191341 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:59.224517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae68968-1357-41f8-9297-ef9d2a649ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:59.225602 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae68968-1357-41f8-9297-ef9d2a649ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:59.226023 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae68968-1357-41f8-9297-ef9d2a649ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:59.226447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae68968-1357-41f8-9297-ef9d2a649ef6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:59.246269 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae68968-1357-41f8-9297-ef9d2a649ef6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:59.250078 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 275/545] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:20:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:20:59.260847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-fae68968-1357-41f8-9297-ef9d2a649ef6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:20:59.260847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:59.260847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:59.260847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:59.260847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:59.260847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:59.260847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:59.260847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:59.260847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:59.260847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:59.260847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:59.260847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:59.260847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:59.260847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:59.260847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:59.260847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:59.261771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:59.261771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:59.261771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:59.261771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:59.261771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:59.261771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:59.261771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:59.261771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:59.261771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:59.261771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:59.261771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:59.261771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:59.261771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:59.261771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:59.261771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:59.261771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:59.262563 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:59.262563 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:59.262563 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:59.262563 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:59.262563 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:59.262563 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:59.262563 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:59.262563 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:59.262563 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:59.262563 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:59.262563 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:20:59.262563 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:59.263188 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 271/546] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:20:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:59.268073 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3688355-452f-474f-8e48-d2f5738b96eb None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:59.283126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae68968-1357-41f8-9297-ef9d2a649ef6 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:59.431377 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:59.450982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3688355-452f-474f-8e48-d2f5738b96eb None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:59.469970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae68968-1357-41f8-9297-ef9d2a649ef6 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:59.496181 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b2a53-e44a-47c4-9147-0e2eee005e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:59.497231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b2a53-e44a-47c4-9147-0e2eee005e83 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:59.497610 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b2a53-e44a-47c4-9147-0e2eee005e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:59.498011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b2a53-e44a-47c4-9147-0e2eee005e83 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:59.498867 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b2a53-e44a-47c4-9147-0e2eee005e83 None admin] RBAC: Adding query filter params (name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:59.499307 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b2a53-e44a-47c4-9147-0e2eee005e83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:59.500776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b2a53-e44a-47c4-9147-0e2eee005e83 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:59.510090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 276/547] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:20:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:59.515999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b2a53-e44a-47c4-9147-0e2eee005e83 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:59.524640 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801c42d-71be-4fe2-a188-b7b0dc39d665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:59.525813 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801c42d-71be-4fe2-a188-b7b0dc39d665 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:59.526254 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801c42d-71be-4fe2-a188-b7b0dc39d665 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:59.526697 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801c42d-71be-4fe2-a188-b7b0dc39d665 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:59.549969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1801c42d-71be-4fe2-a188-b7b0dc39d665 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:20:59.561051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-1801c42d-71be-4fe2-a188-b7b0dc39d665 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:20:59.561051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:59.561051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:59.561051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:59.561051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:59.561051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:59.561051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:59.561051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:59.561051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:59.561051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:59.561051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:59.561051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:59.561051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:20:59.561051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:20:59.561051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:20:59.561051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:20:59.561967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:20:59.561967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:20:59.561967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:20:59.561967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:20:59.561967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:59.561967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:59.561967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:59.561967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:59.561967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:59.561967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:59.561967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:59.561967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:59.561967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:59.561967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:59.561967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:59.561967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:59.562741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:59.562741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:59.562741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:59.562741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:59.562741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:20:59.562741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:20:59.562741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:20:59.562741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:20:59.562741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:20:59.562741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:20:59.562741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:20:59.562741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:20:59.563388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 272/548] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:20:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:20:59.570570 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801c42d-71be-4fe2-a188-b7b0dc39d665 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:59.579302 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:59.697728 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b2a53-e44a-47c4-9147-0e2eee005e83 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:59.764743 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447020e2-30e0-49d4-8124-3d25f3750b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:59.765552 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447020e2-30e0-49d4-8124-3d25f3750b67 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:59.765838 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447020e2-30e0-49d4-8124-3d25f3750b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:59.766114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447020e2-30e0-49d4-8124-3d25f3750b67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:59.767278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801c42d-71be-4fe2-a188-b7b0dc39d665 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:20:59.788019 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447020e2-30e0-49d4-8124-3d25f3750b67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:59.789494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447020e2-30e0-49d4-8124-3d25f3750b67 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:59.816247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-447020e2-30e0-49d4-8124-3d25f3750b67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:59.816247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:20:59.816247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:20:59.816247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:20:59.816247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:20:59.816247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:20:59.816247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:20:59.816247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:20:59.816247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:20:59.816247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:20:59.816247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:20:59.816247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:20:59.816247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:20:59.816247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:20:59.816247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:20:59.817828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:20:59.817828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:20:59.817828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:20:59.817828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:20:59.817828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:20:59.817828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:20:59.817828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:20:59.817828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:20:59.817828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:20:59.817828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:20:59.817828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:20:59.817828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:20:59.817828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:20:59.817828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:20:59.817828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:20:59.817828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:20:59.817828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:20:59.819074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:20:59.819074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:20:59.819074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:20:59.819074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:20:59.819074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 277/549] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:20:59.831821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447020e2-30e0-49d4-8124-3d25f3750b67 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:59.837592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca503e89-7a17-4491-87fe-42761f4b93b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:20:59.838270 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca503e89-7a17-4491-87fe-42761f4b93b5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:20:59.838555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca503e89-7a17-4491-87fe-42761f4b93b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:20:59.838835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca503e89-7a17-4491-87fe-42761f4b93b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:20:59.839626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca503e89-7a17-4491-87fe-42761f4b93b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:20:59.839935 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca503e89-7a17-4491-87fe-42761f4b93b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:20:59.841328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca503e89-7a17-4491-87fe-42761f4b93b5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:20:59.844471 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:20:59.861173 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 273/550] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:20:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:20:59.867508 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca503e89-7a17-4491-87fe-42761f4b93b5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:20:59.981383 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca503e89-7a17-4491-87fe-42761f4b93b5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:00.025339 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8a30c-c1d0-40fc-b388-f576b643f631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:00.026623 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8a30c-c1d0-40fc-b388-f576b643f631 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:00.032230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8a30c-c1d0-40fc-b388-f576b643f631 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:00.032702 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8a30c-c1d0-40fc-b388-f576b643f631 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:00.044430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8a30c-c1d0-40fc-b388-f576b643f631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:00.046176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8a30c-c1d0-40fc-b388-f576b643f631 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:00.065518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:00.079179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-74f8a30c-c1d0-40fc-b388-f576b643f631 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:00.079179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:00.079179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:00.079179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:00.079179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:00.079179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:00.079179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:00.079179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:00.079179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:00.079179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:00.079179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:00.079179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:00.079179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:00.079179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:00.079179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:00.080276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:00.080276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:00.080276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:00.080276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:00.080276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:00.080276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:00.080276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:00.080276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:00.080276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:00.080276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:00.080276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:00.080276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:00.080276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:00.080276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:00.080276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:00.080276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:00.080276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:00.081417 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:00.081417 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:00.081417 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:00.081417 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:00.084921 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 274/551] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:20:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:00.091042 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8a30c-c1d0-40fc-b388-f576b643f631 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:00.098832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447020e2-30e0-49d4-8124-3d25f3750b67 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:00.133987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed941e0d-e02f-469c-a0b5-49e3fdaf9673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:00.134656 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed941e0d-e02f-469c-a0b5-49e3fdaf9673 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:00.134962 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed941e0d-e02f-469c-a0b5-49e3fdaf9673 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:00.135252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed941e0d-e02f-469c-a0b5-49e3fdaf9673 None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:00.135982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed941e0d-e02f-469c-a0b5-49e3fdaf9673 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:00.136291 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed941e0d-e02f-469c-a0b5-49e3fdaf9673 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:00.137661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed941e0d-e02f-469c-a0b5-49e3fdaf9673 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:00.151593 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 278/552] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:00.158705 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed941e0d-e02f-469c-a0b5-49e3fdaf9673 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:00.189740 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:00.255560 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8a30c-c1d0-40fc-b388-f576b643f631 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:00.309183 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee3986-0015-4eb8-ba09-5bf22cf01d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:00.312099 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee3986-0015-4eb8-ba09-5bf22cf01d40 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:00.312599 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee3986-0015-4eb8-ba09-5bf22cf01d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:00.313077 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee3986-0015-4eb8-ba09-5bf22cf01d40 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:00.314035 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee3986-0015-4eb8-ba09-5bf22cf01d40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:00.315311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee3986-0015-4eb8-ba09-5bf22cf01d40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:00.316954 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee3986-0015-4eb8-ba09-5bf22cf01d40 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:00.343566 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 275/553] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:00.349345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee3986-0015-4eb8-ba09-5bf22cf01d40 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:00.387173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed941e0d-e02f-469c-a0b5-49e3fdaf9673 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:00.436269 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b098c-456d-4c35-ba22-0dc1966d7145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:00.437097 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b098c-456d-4c35-ba22-0dc1966d7145 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:00.437387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b098c-456d-4c35-ba22-0dc1966d7145 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:00.437670 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b098c-456d-4c35-ba22-0dc1966d7145 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:00.452134 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b098c-456d-4c35-ba22-0dc1966d7145 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:00.453602 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b098c-456d-4c35-ba22-0dc1966d7145 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:00.491979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-165b098c-456d-4c35-ba22-0dc1966d7145 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 20:21:00.491979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:00.491979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:00.491979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:00.491979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:00.491979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:00.491979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:00.491979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:00.491979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:00.491979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:00.491979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:00.491979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:00.491979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:00.491979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:00.491979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:00.491979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:00.493050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:00.493050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:00.493050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:00.493050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:00.493050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:00.493050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:00.493050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:00.493050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:00.493050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:00.493050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:00.493050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:00.493050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:00.493050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:00.493050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:00.493050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:00.493050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:00.493050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:00.493050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:00.494386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:00.494386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:00.494386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:00.494386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:00.494386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:00.494386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:00.494386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:00.494386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:00.494386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 20:21:00.494386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:00.494386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 279/554] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:21:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:00.501217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b098c-456d-4c35-ba22-0dc1966d7145 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:00.501894 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee3986-0015-4eb8-ba09-5bf22cf01d40 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:00.512802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:00.527058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:00.549876 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e057bb-cb67-4d75-b2a2-8d15b66f876c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:00.550683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e057bb-cb67-4d75-b2a2-8d15b66f876c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:00.550991 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e057bb-cb67-4d75-b2a2-8d15b66f876c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:00.551270 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e057bb-cb67-4d75-b2a2-8d15b66f876c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:00.559551 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e057bb-cb67-4d75-b2a2-8d15b66f876c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:00.561811 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e057bb-cb67-4d75-b2a2-8d15b66f876c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:00.576816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-e1e057bb-cb67-4d75-b2a2-8d15b66f876c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 20:21:00.576816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:00.576816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:00.576816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:00.576816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:00.576816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:00.576816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:00.576816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:00.576816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:00.576816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:00.576816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:00.576816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:00.576816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:00.576816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:00.576816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:00.576816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:00.577800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:00.577800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:00.577800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:00.577800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:00.577800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:00.577800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:00.577800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:00.577800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:00.577800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:00.577800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:00.577800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:00.577800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:00.577800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:00.577800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:00.577800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:00.577800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:00.577800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:00.577800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:00.579490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:00.579490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:00.579490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:00.579490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:00.579490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:00.579490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:00.579490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:00.579490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:00.579490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 20:21:00.579490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:00.579490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 276/555] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 14 20:21:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:00.587267 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e057bb-cb67-4d75-b2a2-8d15b66f876c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:00.674544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b098c-456d-4c35-ba22-0dc1966d7145 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:00.717356 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e057bb-cb67-4d75-b2a2-8d15b66f876c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:00.718729 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c69a29-6582-480f-bed0-7ff6a0ad40f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:00.719655 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c69a29-6582-480f-bed0-7ff6a0ad40f4 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:00.720121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c69a29-6582-480f-bed0-7ff6a0ad40f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:00.720607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c69a29-6582-480f-bed0-7ff6a0ad40f4 None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:00.721635 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c69a29-6582-480f-bed0-7ff6a0ad40f4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:00.722129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c69a29-6582-480f-bed0-7ff6a0ad40f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:00.723797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c69a29-6582-480f-bed0-7ff6a0ad40f4 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:00.749379 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 280/556] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jul 14 20:21:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:00.751155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da505b7-89fd-4b03-91a5-ddaffcd18dc1 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:00.751632 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da505b7-89fd-4b03-91a5-ddaffcd18dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:00.752118 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da505b7-89fd-4b03-91a5-ddaffcd18dc1 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:00.758210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 281/557] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:00.759325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da505b7-89fd-4b03-91a5-ddaffcd18dc1 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:00.764886 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278c6c0-7798-4aeb-accc-c6956a3b947c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:00.765508 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278c6c0-7798-4aeb-accc-c6956a3b947c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:00.765798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278c6c0-7798-4aeb-accc-c6956a3b947c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:00.766082 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278c6c0-7798-4aeb-accc-c6956a3b947c None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:00.766930 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278c6c0-7798-4aeb-accc-c6956a3b947c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:00.767273 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278c6c0-7798-4aeb-accc-c6956a3b947c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:00.769089 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278c6c0-7798-4aeb-accc-c6956a3b947c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:00.795713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 277/558] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 14 20:21:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:00.797458 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb48c79-c6e8-429e-b940-839e81872a1f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:00.798002 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb48c79-c6e8-429e-b940-839e81872a1f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:00.798481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb48c79-c6e8-429e-b940-839e81872a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:00.820268 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-feb48c79-c6e8-429e-b940-839e81872a1f None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:00.821933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-feb48c79-c6e8-429e-b940-839e81872a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:00.846751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:00.855739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:00.889453 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da505b7-89fd-4b03-91a5-ddaffcd18dc1 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:00.928841 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 278/559] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:21:00.930123 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb48c79-c6e8-429e-b940-839e81872a1f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:00.939622 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d72a1-d8d1-4ac6-a4f8-db405d93ec8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:00.940475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d72a1-d8d1-4ac6-a4f8-db405d93ec8e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:00.940832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d72a1-d8d1-4ac6-a4f8-db405d93ec8e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:00.941211 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d72a1-d8d1-4ac6-a4f8-db405d93ec8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:01.054204 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d72a1-d8d1-4ac6-a4f8-db405d93ec8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:01.056217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d72a1-d8d1-4ac6-a4f8-db405d93ec8e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:01.056802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb48c79-c6e8-429e-b940-839e81872a1f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:01.096505 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0a7fb-4afb-41fb-a0ca-f4df0468b24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:01.097554 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0a7fb-4afb-41fb-a0ca-f4df0468b24a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:01.097987 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0a7fb-4afb-41fb-a0ca-f4df0468b24a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:01.098362 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0a7fb-4afb-41fb-a0ca-f4df0468b24a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:01.110948 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0a7fb-4afb-41fb-a0ca-f4df0468b24a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:01.112813 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0a7fb-4afb-41fb-a0ca-f4df0468b24a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:01.133215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-43e0a7fb-4afb-41fb-a0ca-f4df0468b24a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:01.133215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:01.133215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:01.133215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:01.133215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:01.133215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:01.133215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:01.133215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:01.133215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:01.133215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:01.133215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:01.133215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:01.133215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:01.133215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:01.133215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:01.134325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:01.134325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:01.134325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:01.134325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:01.134325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:01.134325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:01.134325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:01.134325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:01.134325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:01.134325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:01.134325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:01.134325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:01.134325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:01.134325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:01.134325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:01.134325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:01.134325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:01.135393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:01.135393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:01.135393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:01.135393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:01.136868 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 279/560] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:01.138140 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0a7fb-4afb-41fb-a0ca-f4df0468b24a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:01.141739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:01.153425 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:01.192179 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 282/561] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:21:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:21:01.193680 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d72a1-d8d1-4ac6-a4f8-db405d93ec8e None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:01.257459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0a7fb-4afb-41fb-a0ca-f4df0468b24a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:01.296458 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c6f09-6b79-474f-85d8-d2dd40cbe3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:01.297259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c6f09-6b79-474f-85d8-d2dd40cbe3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:01.297546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c6f09-6b79-474f-85d8-d2dd40cbe3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:01.297821 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c6f09-6b79-474f-85d8-d2dd40cbe3d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:01.301802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d72a1-d8d1-4ac6-a4f8-db405d93ec8e None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:01.309400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c6f09-6b79-474f-85d8-d2dd40cbe3d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:01.317810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-d77c6f09-6b79-474f-85d8-d2dd40cbe3d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:01.317810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:01.317810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:01.317810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:01.317810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:01.317810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:01.317810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:01.317810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:01.317810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:01.317810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:01.317810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:01.317810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:01.317810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:01.317810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:01.317810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:01.317810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:01.318748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:01.318748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:01.318748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:01.318748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:01.318748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:01.318748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:01.318748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:01.318748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:01.318748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:01.318748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:01.318748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:01.318748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:01.318748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:01.318748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:01.318748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:01.318748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:01.319642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:01.319642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:01.319642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:01.319642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:01.319642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:01.319642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:01.319642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:01.319642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:01.319642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:01.319642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:01.319642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:01.319642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:01.319642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 280/562] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:21:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:01.320330 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb309be1-08db-4249-adb5-fcdaf0064d4a None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:01.320908 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb309be1-08db-4249-adb5-fcdaf0064d4a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:01.320908 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb309be1-08db-4249-adb5-fcdaf0064d4a None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:01.326700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 281/563] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:01.326700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb309be1-08db-4249-adb5-fcdaf0064d4a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:01.347981 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315e645-67b3-4f76-be27-369b832bbdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:01.348653 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315e645-67b3-4f76-be27-369b832bbdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:01.348943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315e645-67b3-4f76-be27-369b832bbdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:01.349217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315e645-67b3-4f76-be27-369b832bbdb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:01.349943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315e645-67b3-4f76-be27-369b832bbdb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:01.350251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315e645-67b3-4f76-be27-369b832bbdb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:01.351652 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315e645-67b3-4f76-be27-369b832bbdb3 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:01.363227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 283/564] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:01.364048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315e645-67b3-4f76-be27-369b832bbdb3 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:01.393575 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:01.399151 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:01.435250 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb309be1-08db-4249-adb5-fcdaf0064d4a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:01.465618 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315e645-67b3-4f76-be27-369b832bbdb3 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:01.469618 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8d0de-742b-403f-8aa9-4608bfbabfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:01.470419 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8d0de-742b-403f-8aa9-4608bfbabfab None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:01.470703 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8d0de-742b-403f-8aa9-4608bfbabfab None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:01.471010 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8d0de-742b-403f-8aa9-4608bfbabfab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:01.481433 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8d0de-742b-403f-8aa9-4608bfbabfab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:01.482954 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8d0de-742b-403f-8aa9-4608bfbabfab None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:01.498393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da0e3f-2a1f-4738-871b-034bdf28ad62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:01.499094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da0e3f-2a1f-4738-871b-034bdf28ad62 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:01.499377 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da0e3f-2a1f-4738-871b-034bdf28ad62 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:01.499648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da0e3f-2a1f-4738-871b-034bdf28ad62 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:01.500391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da0e3f-2a1f-4738-871b-034bdf28ad62 None admin] RBAC: Adding query filter params (name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:01.500692 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da0e3f-2a1f-4738-871b-034bdf28ad62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:01.502053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da0e3f-2a1f-4738-871b-034bdf28ad62 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:01.502170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-d5c8d0de-742b-403f-8aa9-4608bfbabfab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:01.502170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:01.502170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:01.502170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:01.502170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:01.502170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:01.502170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:01.502170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:01.502170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:01.502170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:01.502170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:01.502170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:01.502170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:01.502170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:01.502170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:01.509212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:01.509212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:01.509212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:01.509212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:01.509212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:01.509212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:01.509212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:01.509212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:01.509212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:01.509212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:01.509212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:01.509212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:01.509212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:01.509212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:01.509212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:01.509212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:01.509212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:01.510236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:01.510236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:01.510236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:01.510236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:01.510236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 282/565] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:01.510236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4db40-d7db-4cc6-a8a6-4c5c9ddcd8d6 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:01.510236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4db40-d7db-4cc6-a8a6-4c5c9ddcd8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:01.510236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4db40-d7db-4cc6-a8a6-4c5c9ddcd8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:01.520921 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 284/566] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:21:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:01.522348 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4bad3-fc97-4779-9b41-3125a72fcf98 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:01.522644 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4bad3-fc97-4779-9b41-3125a72fcf98 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:01.522923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4bad3-fc97-4779-9b41-3125a72fcf98 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:01.524434 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 285/567] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:01.524656 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-89f4db40-d7db-4cc6-a8a6-4c5c9ddcd8d6 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:01.525889 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-89f4db40-d7db-4cc6-a8a6-4c5c9ddcd8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:01.527736 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4bad3-fc97-4779-9b41-3125a72fcf98 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:01.594150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 283/568] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:21:01.594892 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4db40-d7db-4cc6-a8a6-4c5c9ddcd8d6 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:01.632284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:01.647607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4bad3-fc97-4779-9b41-3125a72fcf98 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:01.648394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:01.682264 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f14bc0-6645-4622-9f10-b0419742babf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:01.683178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f14bc0-6645-4622-9f10-b0419742babf None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:01.683463 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f14bc0-6645-4622-9f10-b0419742babf None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:01.683738 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f14bc0-6645-4622-9f10-b0419742babf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:01.691393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f14bc0-6645-4622-9f10-b0419742babf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:01.692862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f14bc0-6645-4622-9f10-b0419742babf None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:01.698580 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4db40-d7db-4cc6-a8a6-4c5c9ddcd8d6 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:01.709845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-d7f14bc0-6645-4622-9f10-b0419742babf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:01.709845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:01.709845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:01.709845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:01.709845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:01.709845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:01.709845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:01.709845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:01.709845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:01.709845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:01.709845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:01.709845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:01.709845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:01.709845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:01.709845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:01.710755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:01.710755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:01.710755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:01.710755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:01.710755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:01.710755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:01.710755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:01.710755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:01.710755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:01.710755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:01.710755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:01.710755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:01.710755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:01.710755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:01.710755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:01.710755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:01.710755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:01.711774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:01.711774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:01.711774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:01.711774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:01.711774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:01.711774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:01.711774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:01.711774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:01.711774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 286/569] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:01.712249 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f14bc0-6645-4622-9f10-b0419742babf None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:01.736742 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb5625-47bb-4385-aa1a-8ccc6e19ba9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:01.737411 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb5625-47bb-4385-aa1a-8ccc6e19ba9c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:01.737702 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb5625-47bb-4385-aa1a-8ccc6e19ba9c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:01.737981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb5625-47bb-4385-aa1a-8ccc6e19ba9c None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:01.738705 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb5625-47bb-4385-aa1a-8ccc6e19ba9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:01.739043 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb5625-47bb-4385-aa1a-8ccc6e19ba9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:01.740415 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb5625-47bb-4385-aa1a-8ccc6e19ba9c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:01.752565 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 284/570] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:01.753800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c50251-a6ed-46a6-ac19-3950eefb73f4 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:01.754090 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c50251-a6ed-46a6-ac19-3950eefb73f4 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:01.754369 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c50251-a6ed-46a6-ac19-3950eefb73f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:01.771053 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f3c50251-a6ed-46a6-ac19-3950eefb73f4 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:01.772256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f3c50251-a6ed-46a6-ac19-3950eefb73f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:01.819108 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f14bc0-6645-4622-9f10-b0419742babf None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:01.839789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 285/571] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:21:01.840767 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c50251-a6ed-46a6-ac19-3950eefb73f4 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:01.855261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f74072-1136-493b-bb92-4c79c2f3944d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:01.856003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f74072-1136-493b-bb92-4c79c2f3944d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:01.856292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f74072-1136-493b-bb92-4c79c2f3944d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:01.856579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f74072-1136-493b-bb92-4c79c2f3944d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:01.867577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f74072-1136-493b-bb92-4c79c2f3944d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:01.869033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f74072-1136-493b-bb92-4c79c2f3944d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:01.878365 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:01.885056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:01.889132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-50f74072-1136-493b-bb92-4c79c2f3944d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:01.889132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:01.889132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:01.889132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:01.889132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:01.889132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:01.889132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:01.889132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:01.889132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:01.889132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:01.889132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:01.889132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:01.889132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:01.889132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:01.889132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:01.890770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:01.890770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:01.890770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:01.890770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:01.890770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:01.890770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:01.890770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:01.890770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:01.890770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:01.890770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:01.890770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:01.890770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:01.890770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:01.890770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:01.890770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:01.890770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:01.890770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:01.892014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:01.892014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:01.892014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:01.892014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:01.892014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 287/572] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:01.892014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f74072-1136-493b-bb92-4c79c2f3944d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:01.950266 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c50251-a6ed-46a6-ac19-3950eefb73f4 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:01.985630 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4b391-59d9-4b61-904f-241aab54b23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:01.986568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4b391-59d9-4b61-904f-241aab54b23d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:01.986711 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4b391-59d9-4b61-904f-241aab54b23d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:01.987064 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4b391-59d9-4b61-904f-241aab54b23d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:01.993399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f74072-1136-493b-bb92-4c79c2f3944d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:02.000582 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4b391-59d9-4b61-904f-241aab54b23d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:02.008466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-6eb4b391-59d9-4b61-904f-241aab54b23d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:02.008466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:02.008466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:02.008466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:02.008466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:02.008466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:02.008466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:02.008466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:02.008466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:02.008466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:02.008466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:02.008466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:02.008466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:02.008466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:02.008466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:02.008466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:02.009509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:02.009509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:02.009509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:02.009509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:02.009509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:02.009509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:02.009509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:02.009509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:02.009509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:02.009509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:02.009509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:02.009509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:02.009509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:02.009509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:02.009509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:02.009509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:02.010535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:02.010535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:02.010535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:02.010535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:02.010535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:02.010535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:02.010535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:02.010535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:02.010535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:02.010535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:02.010535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:02.010535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:02.010535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 286/573] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:21:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:02.011277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4b391-59d9-4b61-904f-241aab54b23d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:02.038336 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179413c5-f43c-410a-93df-d93efd87184a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:02.039030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179413c5-f43c-410a-93df-d93efd87184a None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:02.039709 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179413c5-f43c-410a-93df-d93efd87184a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:02.040418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179413c5-f43c-410a-93df-d93efd87184a None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:02.041220 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179413c5-f43c-410a-93df-d93efd87184a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:02.041529 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179413c5-f43c-410a-93df-d93efd87184a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:02.043134 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179413c5-f43c-410a-93df-d93efd87184a None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:02.056286 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 288/574] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:02.057113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179413c5-f43c-410a-93df-d93efd87184a None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:02.109844 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:02.112710 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4b391-59d9-4b61-904f-241aab54b23d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:02.118686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:02.147576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254d15a-c2b0-42a7-b29f-f16efd20eb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:02.148371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254d15a-c2b0-42a7-b29f-f16efd20eb45 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:02.148653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254d15a-c2b0-42a7-b29f-f16efd20eb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:02.148924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254d15a-c2b0-42a7-b29f-f16efd20eb45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:02.155548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254d15a-c2b0-42a7-b29f-f16efd20eb45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:02.157002 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254d15a-c2b0-42a7-b29f-f16efd20eb45 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:02.160213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179413c5-f43c-410a-93df-d93efd87184a None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:02.171470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-d254d15a-c2b0-42a7-b29f-f16efd20eb45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:02.171470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:02.171470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:02.171470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:02.171470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:02.171470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:02.171470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:02.171470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:02.171470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:02.171470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:02.171470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:02.171470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:02.171470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:02.171470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:02.171470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:02.172354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:02.172354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:02.172354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:02.172354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:02.172354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:02.172354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:02.172354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:02.172354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:02.172354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:02.172354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:02.172354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:02.172354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:02.172354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:02.172354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:02.172354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:02.172354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:02.172354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:02.173219 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:02.173219 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:02.173219 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:02.173219 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:02.173219 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:02.173219 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:02.173219 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:02.173219 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:02.173219 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 287/575] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:02.173707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254d15a-c2b0-42a7-b29f-f16efd20eb45 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:02.199907 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87830869-f285-42f5-9236-2a3ae7d02f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:02.200956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87830869-f285-42f5-9236-2a3ae7d02f63 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:02.201233 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87830869-f285-42f5-9236-2a3ae7d02f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:02.201505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87830869-f285-42f5-9236-2a3ae7d02f63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:02.212284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87830869-f285-42f5-9236-2a3ae7d02f63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:02.220035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-87830869-f285-42f5-9236-2a3ae7d02f63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:02.220035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:02.220035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:02.220035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:02.220035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:02.220035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:02.220035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:02.220035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:02.220035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:02.220035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:02.220035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:02.220035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:02.220035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:02.220035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:02.220035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:02.220035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:02.221006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:02.221006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:02.221006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:02.221006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:02.221006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:02.221006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:02.221006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:02.221006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:02.221006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:02.221006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:02.221006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:02.221006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:02.221006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:02.221006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:02.221006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:02.221006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:02.221839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:02.221839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:02.221839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:02.221839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:02.221839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:02.221839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:02.221839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:02.221839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:02.221839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:02.221839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:02.221839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:02.221839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:02.221839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 289/576] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:21:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:02.222577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87830869-f285-42f5-9236-2a3ae7d02f63 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:02.281799 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254d15a-c2b0-42a7-b29f-f16efd20eb45 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:02.319278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce71e89-f142-424a-bd1c-be437c671f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:02.319932 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce71e89-f142-424a-bd1c-be437c671f40 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:02.320220 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce71e89-f142-424a-bd1c-be437c671f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:02.320498 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce71e89-f142-424a-bd1c-be437c671f40 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:02.321225 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce71e89-f142-424a-bd1c-be437c671f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:02.321543 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce71e89-f142-424a-bd1c-be437c671f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:02.322913 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce71e89-f142-424a-bd1c-be437c671f40 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:02.335165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 288/577] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:02.335634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87830869-f285-42f5-9236-2a3ae7d02f63 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:02.335968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce71e89-f142-424a-bd1c-be437c671f40 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:02.356475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:02.360038 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:02.371746 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec48dfd-d8d9-4fd2-b9fc-5b6308e16f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:02.372413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec48dfd-d8d9-4fd2-b9fc-5b6308e16f12 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:02.372734 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec48dfd-d8d9-4fd2-b9fc-5b6308e16f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:02.373011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec48dfd-d8d9-4fd2-b9fc-5b6308e16f12 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:02.373765 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec48dfd-d8d9-4fd2-b9fc-5b6308e16f12 None admin] RBAC: Adding query filter params (name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:02.374071 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec48dfd-d8d9-4fd2-b9fc-5b6308e16f12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:02.375479 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec48dfd-d8d9-4fd2-b9fc-5b6308e16f12 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:02.383886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 290/578] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:21:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:02.384743 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec48dfd-d8d9-4fd2-b9fc-5b6308e16f12 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:02.443640 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce71e89-f142-424a-bd1c-be437c671f40 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:02.476739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580b5aa-f8f8-440f-9c02-c489e6f22e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:02.477624 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580b5aa-f8f8-440f-9c02-c489e6f22e9c None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:02.477908 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580b5aa-f8f8-440f-9c02-c489e6f22e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:02.478182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580b5aa-f8f8-440f-9c02-c489e6f22e9c None admin] PATH_INFO: `/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/aa3bb9f1e24a4e9ebd460e97b3736bc0/roles/ea49c36b4a7f4cfb9048065638d458af` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:02.484808 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec48dfd-d8d9-4fd2-b9fc-5b6308e16f12 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:02.493105 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580b5aa-f8f8-440f-9c02-c489e6f22e9c None admin] RBAC: Authorizing `identity:create_grant(project_id=f33c8ac5154f46a89d1a0dc11fb58cf4, user_id=aa3bb9f1e24a4e9ebd460e97b3736bc0, role_id=ea49c36b4a7f4cfb9048065638d458af)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:02.494548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580b5aa-f8f8-440f-9c02-c489e6f22e9c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:02.507582 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 289/579] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:21:02 2022] PUT /identity/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/aa3bb9f1e24a4e9ebd460e97b3736bc0/roles/ea49c36b4a7f4cfb9048065638d458af => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:21:02.508675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580b5aa-f8f8-440f-9c02-c489e6f22e9c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:02.523685 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5694a-9c57-43f7-82d1-b5269912b606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:02.524578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5694a-9c57-43f7-82d1-b5269912b606 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:02.524776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5694a-9c57-43f7-82d1-b5269912b606 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:02.525002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5694a-9c57-43f7-82d1-b5269912b606 None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:02.525821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b5694a-9c57-43f7-82d1-b5269912b606 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:02.526155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b5694a-9c57-43f7-82d1-b5269912b606 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:02.527680 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b5694a-9c57-43f7-82d1-b5269912b606 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:02.542413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 291/580] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:02.543151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5694a-9c57-43f7-82d1-b5269912b606 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:02.593585 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:02.604094 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:02.650010 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580b5aa-f8f8-440f-9c02-c489e6f22e9c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:02.664251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5694a-9c57-43f7-82d1-b5269912b606 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:02.693464 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49910a-b972-4834-98bb-dd5c7c4d5913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:02.694157 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49910a-b972-4834-98bb-dd5c7c4d5913 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:02.694530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49910a-b972-4834-98bb-dd5c7c4d5913 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:02.694814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49910a-b972-4834-98bb-dd5c7c4d5913 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:02.695731 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49910a-b972-4834-98bb-dd5c7c4d5913 None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:02.696088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49910a-b972-4834-98bb-dd5c7c4d5913 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:02.697608 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49910a-b972-4834-98bb-dd5c7c4d5913 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:02.705601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d29b3-97bd-4255-9123-d10bb4043bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:02.706415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d29b3-97bd-4255-9123-d10bb4043bef None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:02.706700 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d29b3-97bd-4255-9123-d10bb4043bef None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:02.707188 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d29b3-97bd-4255-9123-d10bb4043bef None admin] PATH_INFO: `/v3/users/glance` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:02.707289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 290/581] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:21:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:02.708015 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49910a-b972-4834-98bb-dd5c7c4d5913 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:02.714708 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d29b3-97bd-4255-9123-d10bb4043bef None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:02.716223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d29b3-97bd-4255-9123-d10bb4043bef None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:02.731707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-d77d29b3-97bd-4255-9123-d10bb4043bef None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 20:21:02.731707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:02.731707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:02.731707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:02.731707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:02.731707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:02.731707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:02.731707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:02.731707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:02.731707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:02.731707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:02.731707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:02.731707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:02.731707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:02.731707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:02.731707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:02.732762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:02.732762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:02.732762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:02.732762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:02.732762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:02.732762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:02.732762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:02.732762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:02.732762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:02.732762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:02.732762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:02.732762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:02.732762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:02.732762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:02.732762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:02.732762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:02.732762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:02.732762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:02.733684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:02.733684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:02.733684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:02.733684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:02.733684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:02.733684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:02.733684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:02.733684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:02.733684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 20:21:02.733684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:02.735264 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 292/582] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:21:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:02.736664 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d29b3-97bd-4255-9123-d10bb4043bef None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:02.830067 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49910a-b972-4834-98bb-dd5c7c4d5913 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:02.846295 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d29b3-97bd-4255-9123-d10bb4043bef None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:02.847896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:02.864812 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:02.871351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3dd0f-c513-4d5d-b1e2-2f4d4dc797b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:02.872260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3dd0f-c513-4d5d-b1e2-2f4d4dc797b7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:02.872752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3dd0f-c513-4d5d-b1e2-2f4d4dc797b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:02.873013 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3dd0f-c513-4d5d-b1e2-2f4d4dc797b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:02.884175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f05f3-9812-4e4b-bf46-11892d3f107b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:02.884833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f05f3-9812-4e4b-bf46-11892d3f107b None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:02.885115 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f05f3-9812-4e4b-bf46-11892d3f107b None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:02.885351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3dd0f-c513-4d5d-b1e2-2f4d4dc797b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:02.885451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f05f3-9812-4e4b-bf46-11892d3f107b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:02.886197 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f05f3-9812-4e4b-bf46-11892d3f107b None admin] RBAC: Adding query filter params (role.id=ea49c36b4a7f4cfb9048065638d458af, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=acc23233929144a5b7b3a1733da19828) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:02.886554 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f05f3-9812-4e4b-bf46-11892d3f107b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:02.887673 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3dd0f-c513-4d5d-b1e2-2f4d4dc797b7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:02.887948 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f05f3-9812-4e4b-bf46-11892d3f107b None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:02.894396 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 293/583] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:21:02 2022] GET /identity/v3/role_assignments?user.id=acc23233929144a5b7b3a1733da19828&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=ea49c36b4a7f4cfb9048065638d458af => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:02.895215 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f05f3-9812-4e4b-bf46-11892d3f107b None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:02.919355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-2be3dd0f-c513-4d5d-b1e2-2f4d4dc797b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:02.919355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:02.919355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:02.919355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:02.919355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:02.919355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:02.919355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:02.919355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:02.919355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:02.919355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:02.919355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:02.919355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:02.919355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:02.919355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:02.919355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:02.920390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:02.920390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:02.920390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:02.920390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:02.920390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:02.920390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:02.920390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:02.920390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:02.920390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:02.920390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:02.920390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:02.920390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:02.920390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:02.920390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:02.920390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:02.920390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:02.920390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:02.921276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:02.921276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:02.921276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:02.921276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:02.923893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 291/584] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:02.925017 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3dd0f-c513-4d5d-b1e2-2f4d4dc797b7 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:03.030670 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f05f3-9812-4e4b-bf46-11892d3f107b None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:03.066677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3dd0f-c513-4d5d-b1e2-2f4d4dc797b7 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:03.074589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e330d-910d-4558-87b8-fd316a09face None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:03.075434 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e330d-910d-4558-87b8-fd316a09face None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:03.076321 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e330d-910d-4558-87b8-fd316a09face None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:03.076609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e330d-910d-4558-87b8-fd316a09face None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:03.095189 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e330d-910d-4558-87b8-fd316a09face None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:03.096794 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e330d-910d-4558-87b8-fd316a09face None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:03.112484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68637c2d-ba96-417a-a78f-6b5de0e9e664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:03.113557 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68637c2d-ba96-417a-a78f-6b5de0e9e664 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:03.114038 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68637c2d-ba96-417a-a78f-6b5de0e9e664 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:03.114545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68637c2d-ba96-417a-a78f-6b5de0e9e664 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:03.115639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68637c2d-ba96-417a-a78f-6b5de0e9e664 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:03.116154 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68637c2d-ba96-417a-a78f-6b5de0e9e664 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:03.116323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-ee9e330d-910d-4558-87b8-fd316a09face None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:03.116323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:03.116323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:03.116323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:03.116323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:03.116323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:03.116323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:03.116323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:03.116323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:03.116323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:03.116323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:03.116323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:03.116323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:03.116323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:03.116323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:03.117155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:03.117155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:03.117155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:03.117155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:03.117155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:03.117155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:03.117155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:03.117155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:03.117155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:03.117155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:03.117155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:03.117155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:03.117155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:03.117155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:03.117155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:03.117155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:03.117155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:03.118021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:03.118021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:03.118021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:03.118021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:03.120210 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68637c2d-ba96-417a-a78f-6b5de0e9e664 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:03.123268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 294/585] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:03.124272 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e330d-910d-4558-87b8-fd316a09face None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:03.146841 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:03.147983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 292/586] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jul 14 20:21:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:03.149106 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68637c2d-ba96-417a-a78f-6b5de0e9e664 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:03.211690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:03.254420 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e330d-910d-4558-87b8-fd316a09face None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:03.286870 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68637c2d-ba96-417a-a78f-6b5de0e9e664 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:03.297969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ede316-8eeb-4ebe-8fe0-62ca6cf006fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:03.298985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ede316-8eeb-4ebe-8fe0-62ca6cf006fd None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:03.298985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ede316-8eeb-4ebe-8fe0-62ca6cf006fd None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:03.299241 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ede316-8eeb-4ebe-8fe0-62ca6cf006fd None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:03.299964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ede316-8eeb-4ebe-8fe0-62ca6cf006fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:03.300269 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ede316-8eeb-4ebe-8fe0-62ca6cf006fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:03.301899 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ede316-8eeb-4ebe-8fe0-62ca6cf006fd None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:03.319562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 295/587] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:03.320246 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ede316-8eeb-4ebe-8fe0-62ca6cf006fd None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:03.333502 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6de3c-5f92-40fe-955a-186e80a8f3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:03.334854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6de3c-5f92-40fe-955a-186e80a8f3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:03.335368 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6de3c-5f92-40fe-955a-186e80a8f3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:03.336079 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6de3c-5f92-40fe-955a-186e80a8f3f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:03.336999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6de3c-5f92-40fe-955a-186e80a8f3f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:03.337479 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6de3c-5f92-40fe-955a-186e80a8f3f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:03.339682 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6de3c-5f92-40fe-955a-186e80a8f3f0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:03.355340 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 293/588] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:03.356430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6de3c-5f92-40fe-955a-186e80a8f3f0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:03.418006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:03.451424 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ede316-8eeb-4ebe-8fe0-62ca6cf006fd None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:03.480649 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6de3c-5f92-40fe-955a-186e80a8f3f0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:03.483101 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:03.502570 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ab942-6e81-4af3-bdc2-e284c9594319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:03.504241 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ab942-6e81-4af3-bdc2-e284c9594319 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:03.504736 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ab942-6e81-4af3-bdc2-e284c9594319 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:03.505214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ab942-6e81-4af3-bdc2-e284c9594319 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:03.520942 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d28285-d101-4dde-9821-82b275759ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:03.521794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d28285-d101-4dde-9821-82b275759ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:03.522097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d28285-d101-4dde-9821-82b275759ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:03.522460 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d28285-d101-4dde-9821-82b275759ce3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:03.522837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ab942-6e81-4af3-bdc2-e284c9594319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:03.525082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ab942-6e81-4af3-bdc2-e284c9594319 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:03.531148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d28285-d101-4dde-9821-82b275759ce3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:03.532644 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d28285-d101-4dde-9821-82b275759ce3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:03.547597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-284ab942-6e81-4af3-bdc2-e284c9594319 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:03.547597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:03.547597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:03.547597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:03.547597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:03.547597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:03.547597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:03.547597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:03.547597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:03.547597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:03.547597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:03.547597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:03.547597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:03.547597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:03.547597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:03.548528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:03.548528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:03.548528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:03.548528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:03.548528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:03.548528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:03.548528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:03.548528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:03.548528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:03.548528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:03.548528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:03.548528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:03.548528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:03.548528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:03.548528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:03.548528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:03.548528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:03.549423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:03.549423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:03.549423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:03.549423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:03.549736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-74d28285-d101-4dde-9821-82b275759ce3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 20:21:03.549736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:03.549736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:03.549736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:03.549736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:03.549736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:03.549736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:03.549736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:03.549736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:03.549736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:03.549736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:03.549736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:03.549736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:03.549736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:03.549736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:03.549736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:03.550553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:03.550553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:03.550553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:03.550553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:03.550553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:03.550553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:03.550553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:03.550553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:03.550553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:03.550553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:03.550553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:03.550553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:03.550553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:03.550553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:03.550553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:03.550553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:03.550553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:03.550553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:03.551525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:03.551525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:03.551525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:03.551525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:03.551525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:03.551525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:03.551525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:03.551525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:03.551525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 20:21:03.551525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:03.554513 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 296/589] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:03.555250 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 294/590] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:21:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:03.555843 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ab942-6e81-4af3-bdc2-e284c9594319 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:03.556119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d28285-d101-4dde-9821-82b275759ce3 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:03.677527 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d28285-d101-4dde-9821-82b275759ce3 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:03.692030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ab942-6e81-4af3-bdc2-e284c9594319 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:03.719429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635d5cb-1c80-4f0e-b377-e8e4dd98c771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:03.720239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635d5cb-1c80-4f0e-b377-e8e4dd98c771 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:03.720493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635d5cb-1c80-4f0e-b377-e8e4dd98c771 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:03.720775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635d5cb-1c80-4f0e-b377-e8e4dd98c771 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:03.721616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635d5cb-1c80-4f0e-b377-e8e4dd98c771 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:03.721969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635d5cb-1c80-4f0e-b377-e8e4dd98c771 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:03.723501 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635d5cb-1c80-4f0e-b377-e8e4dd98c771 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:03.727685 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:03.733382 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f992164-5680-48a0-8cfb-245180b091ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:03.734674 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f992164-5680-48a0-8cfb-245180b091ff None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:03.735190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f992164-5680-48a0-8cfb-245180b091ff None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:03.735703 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f992164-5680-48a0-8cfb-245180b091ff None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:03.744053 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 295/591] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jul 14 20:21:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:03.744468 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f992164-5680-48a0-8cfb-245180b091ff None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:03.744972 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635d5cb-1c80-4f0e-b377-e8e4dd98c771 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:03.746483 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f992164-5680-48a0-8cfb-245180b091ff None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:03.762467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-3f992164-5680-48a0-8cfb-245180b091ff None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 20:21:03.762467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:03.762467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:03.762467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:03.762467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:03.762467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:03.762467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:03.762467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:03.762467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:03.762467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:03.762467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:03.762467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:03.762467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:03.762467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:03.762467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:03.762467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:03.764020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:03.764020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:03.764020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:03.764020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:03.764020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:03.764020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:03.764020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:03.764020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:03.764020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:03.764020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:03.764020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:03.764020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:03.764020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:03.764020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:03.764020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:03.764020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:03.764020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:03.764020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:03.764982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:03.764982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:03.764982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:03.764982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:03.764982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:03.764982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:03.764982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:03.764982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:03.764982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 20:21:03.764982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:03.767690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:03.769142 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 297/592] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:21:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:03.770314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f992164-5680-48a0-8cfb-245180b091ff None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:03.854072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635d5cb-1c80-4f0e-b377-e8e4dd98c771 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:03.880863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f992164-5680-48a0-8cfb-245180b091ff None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:03.892538 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da364bf-2f0f-42ec-895a-5954fae3bdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:03.893218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da364bf-2f0f-42ec-895a-5954fae3bdd5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:03.893506 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da364bf-2f0f-42ec-895a-5954fae3bdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:03.894029 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da364bf-2f0f-42ec-895a-5954fae3bdd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:03.894828 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da364bf-2f0f-42ec-895a-5954fae3bdd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:03.895297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da364bf-2f0f-42ec-895a-5954fae3bdd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:03.896579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da364bf-2f0f-42ec-895a-5954fae3bdd5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:03.909429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 296/593] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:03.910429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da364bf-2f0f-42ec-895a-5954fae3bdd5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:03.921071 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1c794-970e-4b18-8bc3-d37f5b6f9436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:03.922676 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1c794-970e-4b18-8bc3-d37f5b6f9436 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:03.923189 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1c794-970e-4b18-8bc3-d37f5b6f9436 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:03.923686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1c794-970e-4b18-8bc3-d37f5b6f9436 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:03.936613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1c794-970e-4b18-8bc3-d37f5b6f9436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:03.939230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1c794-970e-4b18-8bc3-d37f5b6f9436 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:03.959856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-d5d1c794-970e-4b18-8bc3-d37f5b6f9436 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:03.959856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:03.959856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:03.959856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:03.959856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:03.959856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:03.959856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:03.959856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:03.959856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:03.959856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:03.959856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:03.959856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:03.959856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:03.959856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:03.959856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:03.960793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:03.960793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:03.960793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:03.960793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:03.960793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:03.960793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:03.960793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:03.960793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:03.960793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:03.960793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:03.960793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:03.960793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:03.960793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:03.960793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:03.960793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:03.960793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:03.960793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:03.961676 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:03.961676 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:03.961676 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:03.961676 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:03.964053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 298/594] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:03.965185 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1c794-970e-4b18-8bc3-d37f5b6f9436 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:04.007100 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:04.022564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:04.023804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da364bf-2f0f-42ec-895a-5954fae3bdd5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:04.062082 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b33424-b3f8-48fe-98fb-b98ca50323e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:04.062917 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b33424-b3f8-48fe-98fb-b98ca50323e9 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:04.063192 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b33424-b3f8-48fe-98fb-b98ca50323e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:04.063472 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b33424-b3f8-48fe-98fb-b98ca50323e9 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:04.064319 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b33424-b3f8-48fe-98fb-b98ca50323e9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:04.064652 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b33424-b3f8-48fe-98fb-b98ca50323e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:04.066193 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b33424-b3f8-48fe-98fb-b98ca50323e9 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:04.087599 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 297/595] 10.222.0.52 () {56 vars in 1139 bytes} [Thu Jul 14 20:21:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:04.092219 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b33424-b3f8-48fe-98fb-b98ca50323e9 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:04.095521 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1c794-970e-4b18-8bc3-d37f5b6f9436 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:04.135796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9b415-462d-4a5e-b28b-166ec91266bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:04.136874 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9b415-462d-4a5e-b28b-166ec91266bf None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:04.137349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9b415-462d-4a5e-b28b-166ec91266bf None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:04.137761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9b415-462d-4a5e-b28b-166ec91266bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:04.146422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9b415-462d-4a5e-b28b-166ec91266bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:04.148190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9b415-462d-4a5e-b28b-166ec91266bf None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:04.163452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-d9d9b415-462d-4a5e-b28b-166ec91266bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:04.163452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:04.163452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:04.163452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:04.163452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:04.163452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:04.163452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:04.163452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:04.163452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:04.163452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:04.163452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:04.163452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:04.163452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:04.163452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:04.163452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:04.164559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:04.164559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:04.164559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:04.164559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:04.164559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:04.164559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:04.164559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:04.164559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:04.164559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:04.164559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:04.164559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:04.164559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:04.164559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:04.164559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:04.164559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:04.164559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:04.164559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:04.165471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:04.165471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:04.165471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:04.165471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:04.165471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:04.165471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:04.165471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:04.165471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:04.167555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 299/596] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:04.168582 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9b415-462d-4a5e-b28b-166ec91266bf None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:04.228805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b33424-b3f8-48fe-98fb-b98ca50323e9 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:04.271096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9123a5e-38b9-402d-8169-c42828b420bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:04.272062 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9123a5e-38b9-402d-8169-c42828b420bb None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:04.272550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9123a5e-38b9-402d-8169-c42828b420bb None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:04.272995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9123a5e-38b9-402d-8169-c42828b420bb None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:04.273885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9123a5e-38b9-402d-8169-c42828b420bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:04.274380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9123a5e-38b9-402d-8169-c42828b420bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:04.275947 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9123a5e-38b9-402d-8169-c42828b420bb None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:04.283072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:04.283458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9b415-462d-4a5e-b28b-166ec91266bf None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:04.285959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:04.306124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 298/597] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:04.307372 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9123a5e-38b9-402d-8169-c42828b420bb None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:04.332047 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e6e4c-c0e8-44a0-8049-6a20fe8d3b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:04.332654 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e6e4c-c0e8-44a0-8049-6a20fe8d3b9d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:04.332947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e6e4c-c0e8-44a0-8049-6a20fe8d3b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:04.333229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e6e4c-c0e8-44a0-8049-6a20fe8d3b9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:04.344440 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e6e4c-c0e8-44a0-8049-6a20fe8d3b9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:04.345898 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e6e4c-c0e8-44a0-8049-6a20fe8d3b9d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:04.364984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-770e6e4c-c0e8-44a0-8049-6a20fe8d3b9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:04.364984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:04.364984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:04.364984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:04.364984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:04.364984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:04.364984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:04.364984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:04.364984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:04.364984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:04.364984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:04.364984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:04.364984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:04.364984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:04.364984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:04.365924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:04.365924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:04.365924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:04.365924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:04.365924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:04.365924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:04.365924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:04.365924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:04.365924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:04.365924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:04.365924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:04.365924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:04.365924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:04.365924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:04.365924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:04.365924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:04.365924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:04.366804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:04.366804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:04.366804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:04.366804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:04.366804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 300/598] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:04.371534 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e6e4c-c0e8-44a0-8049-6a20fe8d3b9d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:04.423547 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9123a5e-38b9-402d-8169-c42828b420bb None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:04.460196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4e10e-7f5e-464e-b5e8-b01ad97ca87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:04.461107 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4e10e-7f5e-464e-b5e8-b01ad97ca87d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:04.461533 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4e10e-7f5e-464e-b5e8-b01ad97ca87d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:04.461972 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4e10e-7f5e-464e-b5e8-b01ad97ca87d None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:04.462911 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4e10e-7f5e-464e-b5e8-b01ad97ca87d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:04.463388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4e10e-7f5e-464e-b5e8-b01ad97ca87d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:04.464967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4e10e-7f5e-464e-b5e8-b01ad97ca87d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:04.478186 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 299/599] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:04.479310 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4e10e-7f5e-464e-b5e8-b01ad97ca87d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:04.532277 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e6e4c-c0e8-44a0-8049-6a20fe8d3b9d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:04.586538 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8aab9-4c30-46e8-a0e2-259a126f3255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:04.587901 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8aab9-4c30-46e8-a0e2-259a126f3255 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:04.588188 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8aab9-4c30-46e8-a0e2-259a126f3255 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:04.588494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8aab9-4c30-46e8-a0e2-259a126f3255 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:04.597392 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8aab9-4c30-46e8-a0e2-259a126f3255 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:04.599171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8aab9-4c30-46e8-a0e2-259a126f3255 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:04.600798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:04.617144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-76a8aab9-4c30-46e8-a0e2-259a126f3255 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:04.617144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:04.617144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:04.617144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:04.617144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:04.617144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:04.617144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:04.617144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:04.617144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:04.617144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:04.617144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:04.617144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:04.617144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:04.617144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:04.617144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:04.618283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:04.618283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:04.618283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:04.618283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:04.618283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:04.618283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:04.618283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:04.618283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:04.618283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:04.618283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:04.618283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:04.618283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:04.618283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:04.618283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:04.618283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:04.618283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:04.618283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:04.619288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:04.619288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:04.619288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:04.619288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:04.619288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:04.619288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:04.619288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:04.619288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:04.621191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 301/600] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:04.622202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8aab9-4c30-46e8-a0e2-259a126f3255 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:04.646781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:04.666019 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4e10e-7f5e-464e-b5e8-b01ad97ca87d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:04.704488 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e296c-271e-4b7f-b5e7-054d4d6e57ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:04.705262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e296c-271e-4b7f-b5e7-054d4d6e57ab None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:04.705550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e296c-271e-4b7f-b5e7-054d4d6e57ab None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:04.705824 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e296c-271e-4b7f-b5e7-054d4d6e57ab None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:04.706551 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e296c-271e-4b7f-b5e7-054d4d6e57ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:04.706857 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e296c-271e-4b7f-b5e7-054d4d6e57ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:04.708276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e296c-271e-4b7f-b5e7-054d4d6e57ab None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:04.725572 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 300/601] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:04.725572 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e296c-271e-4b7f-b5e7-054d4d6e57ab None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:04.751187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8aab9-4c30-46e8-a0e2-259a126f3255 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:04.796339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab05ff-5932-4a16-979f-853a35ad07ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:04.797388 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab05ff-5932-4a16-979f-853a35ad07ab None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:04.797787 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab05ff-5932-4a16-979f-853a35ad07ab None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:04.798064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab05ff-5932-4a16-979f-853a35ad07ab None admin] PATH_INFO: `/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/4b7a38b4a7754037a00d11dafc88645b/roles/ea49c36b4a7f4cfb9048065638d458af` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:04.814266 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab05ff-5932-4a16-979f-853a35ad07ab None admin] RBAC: Authorizing `identity:create_grant(project_id=f33c8ac5154f46a89d1a0dc11fb58cf4, user_id=4b7a38b4a7754037a00d11dafc88645b, role_id=ea49c36b4a7f4cfb9048065638d458af)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:04.815893 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab05ff-5932-4a16-979f-853a35ad07ab None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:04.830235 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 302/602] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:21:04 2022] PUT /identity/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/4b7a38b4a7754037a00d11dafc88645b/roles/ea49c36b4a7f4cfb9048065638d458af => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:21:04.831679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15349dbc-7a52-46eb-bf6e-13365c7ae22a None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:04.831969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15349dbc-7a52-46eb-bf6e-13365c7ae22a None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:04.832247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15349dbc-7a52-46eb-bf6e-13365c7ae22a None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:04.833790 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 303/603] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:04.838864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15349dbc-7a52-46eb-bf6e-13365c7ae22a None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:04.839400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e296c-271e-4b7f-b5e7-054d4d6e57ab None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:04.865309 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:04.889837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ceede-aa2c-4458-9d43-4e26ddde35b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:04.891798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ceede-aa2c-4458-9d43-4e26ddde35b3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:04.892319 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ceede-aa2c-4458-9d43-4e26ddde35b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:04.892772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ceede-aa2c-4458-9d43-4e26ddde35b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:04.894732 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ceede-aa2c-4458-9d43-4e26ddde35b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:04.895277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ceede-aa2c-4458-9d43-4e26ddde35b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:04.897817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ceede-aa2c-4458-9d43-4e26ddde35b3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:04.917046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 301/604] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:04.918729 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b002795-2152-432d-8d1a-ebb1812cb302 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:04.919295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b002795-2152-432d-8d1a-ebb1812cb302 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:04.920321 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b002795-2152-432d-8d1a-ebb1812cb302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:04.925600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:04.949609 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-2b002795-2152-432d-8d1a-ebb1812cb302 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:04.952505 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2b002795-2152-432d-8d1a-ebb1812cb302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:04.978096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15349dbc-7a52-46eb-bf6e-13365c7ae22a None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:05.036187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f80ef-e408-49bf-b209-12ecb268ea1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:05.037063 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f80ef-e408-49bf-b209-12ecb268ea1e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:05.037354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f80ef-e408-49bf-b209-12ecb268ea1e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:05.037634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f80ef-e408-49bf-b209-12ecb268ea1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:05.046340 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f80ef-e408-49bf-b209-12ecb268ea1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:05.047812 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f80ef-e408-49bf-b209-12ecb268ea1e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:05.052591 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 302/605] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:21:05.053710 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b002795-2152-432d-8d1a-ebb1812cb302 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:05.063562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-957f80ef-e408-49bf-b209-12ecb268ea1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:05.063562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:05.063562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:05.063562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:05.063562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:05.063562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:05.063562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:05.063562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:05.063562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:05.063562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:05.063562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:05.063562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:05.063562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:05.063562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:05.063562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:05.064532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:05.064532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:05.064532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:05.064532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:05.064532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:05.064532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:05.064532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:05.064532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:05.064532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:05.064532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:05.064532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:05.064532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:05.064532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:05.064532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:05.064532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:05.064532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:05.064532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:05.065425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:05.065425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:05.065425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:05.065425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:05.065425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:05.065425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:05.065425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:05.065425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:05.065425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 304/606] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:05.066393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc9dd2-8827-4e86-a7d3-f1d4dba81ba9 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:05.066393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc9dd2-8827-4e86-a7d3-f1d4dba81ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:05.066639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc9dd2-8827-4e86-a7d3-f1d4dba81ba9 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:05.069672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 305/607] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:05.073838 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc9dd2-8827-4e86-a7d3-f1d4dba81ba9 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:05.180548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b002795-2152-432d-8d1a-ebb1812cb302 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:05.188799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc9dd2-8827-4e86-a7d3-f1d4dba81ba9 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:05.191503 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:05.215989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:05.220463 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f22fe-137b-40cc-892a-4a060e74177f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:05.221372 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f22fe-137b-40cc-892a-4a060e74177f None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:05.221654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f22fe-137b-40cc-892a-4a060e74177f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:05.221931 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f22fe-137b-40cc-892a-4a060e74177f None admin] PATH_INFO: `/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/01b4142ea6f142df9b1314532c961f75/roles/ea49c36b4a7f4cfb9048065638d458af` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:05.230363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6597e37-60dd-473f-bfdf-57b9146e8752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:05.231125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6597e37-60dd-473f-bfdf-57b9146e8752 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:05.231354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6597e37-60dd-473f-bfdf-57b9146e8752 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:05.231623 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6597e37-60dd-473f-bfdf-57b9146e8752 None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:05.232461 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6597e37-60dd-473f-bfdf-57b9146e8752 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:05.232772 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6597e37-60dd-473f-bfdf-57b9146e8752 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:05.234221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6597e37-60dd-473f-bfdf-57b9146e8752 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:05.238628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f22fe-137b-40cc-892a-4a060e74177f None admin] RBAC: Authorizing `identity:create_grant(project_id=f33c8ac5154f46a89d1a0dc11fb58cf4, user_id=01b4142ea6f142df9b1314532c961f75, role_id=ea49c36b4a7f4cfb9048065638d458af)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:05.240121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f22fe-137b-40cc-892a-4a060e74177f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:05.248228 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 306/608] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:05.251557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6597e37-60dd-473f-bfdf-57b9146e8752 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:05.259186 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 303/609] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:21:05 2022] PUT /identity/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/01b4142ea6f142df9b1314532c961f75/roles/ea49c36b4a7f4cfb9048065638d458af => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:21:05.260462 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921171c2-69fe-4f76-90b9-60be389db88d None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:05.260749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921171c2-69fe-4f76-90b9-60be389db88d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:05.261020 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921171c2-69fe-4f76-90b9-60be389db88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:05.287400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-921171c2-69fe-4f76-90b9-60be389db88d None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:05.289001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-921171c2-69fe-4f76-90b9-60be389db88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:05.374168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 304/610] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:21:05.375195 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6597e37-60dd-473f-bfdf-57b9146e8752 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:05.379313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921171c2-69fe-4f76-90b9-60be389db88d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:05.415340 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa77e8e-3e02-4b2f-8bcf-247d4f408e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:05.416561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa77e8e-3e02-4b2f-8bcf-247d4f408e3d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:05.417062 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa77e8e-3e02-4b2f-8bcf-247d4f408e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:05.417570 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa77e8e-3e02-4b2f-8bcf-247d4f408e3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:05.430180 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa77e8e-3e02-4b2f-8bcf-247d4f408e3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:05.439198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-7aa77e8e-3e02-4b2f-8bcf-247d4f408e3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:05.439198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:05.439198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:05.439198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:05.439198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:05.439198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:05.439198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:05.439198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:05.439198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:05.439198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:05.439198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:05.439198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:05.439198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:05.439198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:05.439198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:05.439198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:05.440202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:05.440202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:05.440202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:05.440202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:05.440202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:05.440202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:05.440202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:05.440202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:05.440202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:05.440202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:05.440202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:05.440202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:05.440202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:05.440202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:05.440202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:05.440202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:05.441030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:05.441030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:05.441030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:05.441030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:05.441030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:05.441030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:05.441030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:05.441030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:05.441030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:05.441030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:05.441030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:05.441030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:05.443970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 307/611] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:21:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:05.445215 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa77e8e-3e02-4b2f-8bcf-247d4f408e3d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:05.516803 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:05.526648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:05.535850 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921171c2-69fe-4f76-90b9-60be389db88d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:05.577658 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad75be7-2938-4a2a-8005-61736d31da12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:05.578919 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad75be7-2938-4a2a-8005-61736d31da12 None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:05.579341 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa77e8e-3e02-4b2f-8bcf-247d4f408e3d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:05.579691 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad75be7-2938-4a2a-8005-61736d31da12 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:05.580189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad75be7-2938-4a2a-8005-61736d31da12 None admin] PATH_INFO: `/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/f88257850ee34ad895208ab935b6e643/roles/ea49c36b4a7f4cfb9048065638d458af` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:05.597273 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad75be7-2938-4a2a-8005-61736d31da12 None admin] RBAC: Authorizing `identity:create_grant(project_id=f33c8ac5154f46a89d1a0dc11fb58cf4, user_id=f88257850ee34ad895208ab935b6e643, role_id=ea49c36b4a7f4cfb9048065638d458af)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:05.599125 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad75be7-2938-4a2a-8005-61736d31da12 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:05.616535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 305/612] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:21:05 2022] PUT /identity/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/f88257850ee34ad895208ab935b6e643/roles/ea49c36b4a7f4cfb9048065638d458af => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:21:05.617200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e05033-39e3-4601-8f02-62828812f4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:05.618016 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad75be7-2938-4a2a-8005-61736d31da12 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:05.618126 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e05033-39e3-4601-8f02-62828812f4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:05.618387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e05033-39e3-4601-8f02-62828812f4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:05.618667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e05033-39e3-4601-8f02-62828812f4c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:05.644161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e05033-39e3-4601-8f02-62828812f4c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:05.657603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-85e05033-39e3-4601-8f02-62828812f4c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:05.657603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:05.657603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:05.657603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:05.657603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:05.657603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:05.657603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:05.657603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:05.657603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:05.657603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:05.657603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:05.657603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:05.657603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:05.657603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:05.657603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:05.657603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:05.658688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:05.658688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:05.658688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:05.658688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:05.658688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:05.658688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:05.658688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:05.658688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:05.658688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:05.658688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:05.658688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:05.658688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:05.658688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:05.658688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:05.658688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:05.658688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:05.660061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:05.660061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:05.660061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:05.660061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:05.660061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:05.660061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:05.660061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:05.660061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:05.660061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:05.660061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:05.660061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:05.660061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:05.665785 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 308/613] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:21:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:05.674221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e05033-39e3-4601-8f02-62828812f4c7 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:05.843739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e05033-39e3-4601-8f02-62828812f4c7 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:05.847335 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:05.855445 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad75be7-2938-4a2a-8005-61736d31da12 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:05.884193 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e20fea-3cbb-4a8d-9c47-0bbdfb550546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:05.885235 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e20fea-3cbb-4a8d-9c47-0bbdfb550546 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:05.885621 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e20fea-3cbb-4a8d-9c47-0bbdfb550546 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:05.886050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e20fea-3cbb-4a8d-9c47-0bbdfb550546 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:05.887049 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e20fea-3cbb-4a8d-9c47-0bbdfb550546 None admin] RBAC: Adding query filter params (name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:05.887502 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e20fea-3cbb-4a8d-9c47-0bbdfb550546 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:05.889127 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e20fea-3cbb-4a8d-9c47-0bbdfb550546 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:05.896473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92ca85-667b-4c2f-babe-82a21249bfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:05.897162 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92ca85-667b-4c2f-babe-82a21249bfad None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:05.897444 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92ca85-667b-4c2f-babe-82a21249bfad None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:05.897718 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92ca85-667b-4c2f-babe-82a21249bfad None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:05.898477 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92ca85-667b-4c2f-babe-82a21249bfad None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:05.898783 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92ca85-667b-4c2f-babe-82a21249bfad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:05.900212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92ca85-667b-4c2f-babe-82a21249bfad None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:05.900335 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 309/614] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:21:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:05.910390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:05.911659 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e20fea-3cbb-4a8d-9c47-0bbdfb550546 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:05.916508 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 306/615] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:21:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:05.925229 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92ca85-667b-4c2f-babe-82a21249bfad None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:06.079013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e20fea-3cbb-4a8d-9c47-0bbdfb550546 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:06.120471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c4f97-d0d7-4a68-8140-ecda3eeaf490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:06.121205 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c4f97-d0d7-4a68-8140-ecda3eeaf490 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:06.121490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c4f97-d0d7-4a68-8140-ecda3eeaf490 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:06.121768 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c4f97-d0d7-4a68-8140-ecda3eeaf490 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:06.143488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c4f97-d0d7-4a68-8140-ecda3eeaf490 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:06.148524 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c4f97-d0d7-4a68-8140-ecda3eeaf490 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:06.157432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92ca85-667b-4c2f-babe-82a21249bfad None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:06.185100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-d64c4f97-d0d7-4a68-8140-ecda3eeaf490 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:06.185100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:06.185100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:06.185100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:06.185100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:06.185100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:06.185100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:06.185100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:06.185100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:06.185100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:06.185100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:06.185100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:06.185100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:06.185100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:06.185100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:06.186070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:06.186070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:06.186070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:06.186070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:06.186070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:06.186070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:06.186070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:06.186070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:06.186070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:06.186070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:06.186070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:06.186070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:06.186070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:06.186070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:06.186070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:06.186070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:06.186070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:06.186991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:06.186991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:06.186991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:06.186991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:06.189513 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 310/616] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:06.193246 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c4f97-d0d7-4a68-8140-ecda3eeaf490 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:06.228929 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:06.243679 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da2fb9-831e-4b64-83a6-aee31348e3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:06.244504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da2fb9-831e-4b64-83a6-aee31348e3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:06.244822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da2fb9-831e-4b64-83a6-aee31348e3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:06.245077 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da2fb9-831e-4b64-83a6-aee31348e3e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:06.261917 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da2fb9-831e-4b64-83a6-aee31348e3e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:06.271579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da2fb9-831e-4b64-83a6-aee31348e3e3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:06.308212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c4f97-d0d7-4a68-8140-ecda3eeaf490 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:06.316400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-73da2fb9-831e-4b64-83a6-aee31348e3e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:06.316400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:06.316400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:06.316400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:06.316400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:06.316400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:06.316400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:06.316400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:06.316400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:06.316400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:06.316400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:06.316400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:06.316400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:06.316400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:06.316400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:06.317345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:06.317345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:06.317345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:06.317345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:06.317345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:06.317345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:06.317345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:06.317345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:06.317345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:06.317345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:06.317345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:06.317345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:06.317345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:06.317345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:06.317345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:06.317345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:06.317345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:06.318241 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:06.318241 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:06.318241 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:06.318241 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:06.318241 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 307/617] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:06.323422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da2fb9-831e-4b64-83a6-aee31348e3e3 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:06.348549 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873973bb-96b8-4c7d-8b8e-a570d30820b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:06.349416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873973bb-96b8-4c7d-8b8e-a570d30820b5 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:06.349764 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873973bb-96b8-4c7d-8b8e-a570d30820b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:06.350082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873973bb-96b8-4c7d-8b8e-a570d30820b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:06.350866 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873973bb-96b8-4c7d-8b8e-a570d30820b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:06.351241 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873973bb-96b8-4c7d-8b8e-a570d30820b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:06.352731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873973bb-96b8-4c7d-8b8e-a570d30820b5 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:06.379697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 311/618] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:06.386332 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873973bb-96b8-4c7d-8b8e-a570d30820b5 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:06.444147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:06.478276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:06.504240 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da2fb9-831e-4b64-83a6-aee31348e3e3 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:06.509321 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873973bb-96b8-4c7d-8b8e-a570d30820b5 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:06.555226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8edbc-8110-499d-a979-857db4f8410d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:06.556162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8edbc-8110-499d-a979-857db4f8410d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:06.556450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8edbc-8110-499d-a979-857db4f8410d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:06.556728 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8edbc-8110-499d-a979-857db4f8410d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:06.559325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011154a4-20e5-4ce3-ac8b-44f4791e78a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:06.560201 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011154a4-20e5-4ce3-ac8b-44f4791e78a0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:06.560489 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011154a4-20e5-4ce3-ac8b-44f4791e78a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:06.560767 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011154a4-20e5-4ce3-ac8b-44f4791e78a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:06.561502 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011154a4-20e5-4ce3-ac8b-44f4791e78a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:06.561810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011154a4-20e5-4ce3-ac8b-44f4791e78a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:06.563231 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011154a4-20e5-4ce3-ac8b-44f4791e78a0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:06.564834 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8edbc-8110-499d-a979-857db4f8410d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:06.566293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8edbc-8110-499d-a979-857db4f8410d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:06.583125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-1cd8edbc-8110-499d-a979-857db4f8410d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 20:21:06.583125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:06.583125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:06.583125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:06.583125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:06.583125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:06.583125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:06.583125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:06.583125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:06.583125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:06.583125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:06.583125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:06.583125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:06.583125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:06.583125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:06.583125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:06.584271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:06.584271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:06.584271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:06.584271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:06.584271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:06.584271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:06.584271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:06.584271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:06.584271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:06.584271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:06.584271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:06.584271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:06.584271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:06.584271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:06.584271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:06.584271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:06.584271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:06.584271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:06.585232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:06.585232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:06.585232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:06.585232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:06.585232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:06.585232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:06.585232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:06.585232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:06.585232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 20:21:06.585232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:06.585232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 312/619] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:21:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:21:06.591192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8edbc-8110-499d-a979-857db4f8410d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:06.594194 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 308/620] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:06.599724 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011154a4-20e5-4ce3-ac8b-44f4791e78a0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:06.722840 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8edbc-8110-499d-a979-857db4f8410d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:06.762622 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036f554-73e2-4afb-aa45-0cafd3ba4c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:06.764055 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036f554-73e2-4afb-aa45-0cafd3ba4c37 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:06.766959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036f554-73e2-4afb-aa45-0cafd3ba4c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:06.766959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036f554-73e2-4afb-aa45-0cafd3ba4c37 None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:06.766959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036f554-73e2-4afb-aa45-0cafd3ba4c37 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:06.766959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036f554-73e2-4afb-aa45-0cafd3ba4c37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:06.767400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036f554-73e2-4afb-aa45-0cafd3ba4c37 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:06.789315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 313/621] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jul 14 20:21:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:06.791294 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:06.796593 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036f554-73e2-4afb-aa45-0cafd3ba4c37 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:06.846818 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011154a4-20e5-4ce3-ac8b-44f4791e78a0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:06.883381 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:06.912706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7216f2e-f1dc-4fba-ac04-d80cf38258fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:06.913499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7216f2e-f1dc-4fba-ac04-d80cf38258fa None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:06.913782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7216f2e-f1dc-4fba-ac04-d80cf38258fa None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:06.914054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7216f2e-f1dc-4fba-ac04-d80cf38258fa None admin] PATH_INFO: `/v3/users/nova` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:06.925113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7216f2e-f1dc-4fba-ac04-d80cf38258fa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:06.926811 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7216f2e-f1dc-4fba-ac04-d80cf38258fa None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:06.947912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-a7216f2e-f1dc-4fba-ac04-d80cf38258fa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 20:21:06.947912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:06.947912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:06.947912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:06.947912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:06.947912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:06.947912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:06.947912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:06.947912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:06.947912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:06.947912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:06.947912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:06.947912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:06.947912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:06.947912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:06.947912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:06.948929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:06.948929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:06.948929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:06.948929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:06.948929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:06.948929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:06.948929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:06.948929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:06.948929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:06.948929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:06.948929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:06.948929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:06.948929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:06.948929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:06.948929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:06.948929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:06.948929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:06.948929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:06.949879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:06.949879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:06.949879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:06.949879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:06.949879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:06.949879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:06.949879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:06.949879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:06.949879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 20:21:06.949879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:06.949879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 309/622] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 14 20:21:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:21:06.960080 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7216f2e-f1dc-4fba-ac04-d80cf38258fa None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:07.004975 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036f554-73e2-4afb-aa45-0cafd3ba4c37 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:07.076578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7216f2e-f1dc-4fba-ac04-d80cf38258fa None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:07.116534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd06675-816a-4123-8091-a7431b489ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:07.117233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd06675-816a-4123-8091-a7431b489ebb None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:07.117522 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd06675-816a-4123-8091-a7431b489ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:07.117803 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd06675-816a-4123-8091-a7431b489ebb None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:07.118674 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd06675-816a-4123-8091-a7431b489ebb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:07.119031 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd06675-816a-4123-8091-a7431b489ebb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:07.120528 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd06675-816a-4123-8091-a7431b489ebb None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:07.125615 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4810c8f-6878-4029-9fd4-76feadeadce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:07.127161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4810c8f-6878-4029-9fd4-76feadeadce7 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:07.127655 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4810c8f-6878-4029-9fd4-76feadeadce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:07.128110 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4810c8f-6878-4029-9fd4-76feadeadce7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:07.142254 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 310/623] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 14 20:21:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:07.143589 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405d501-44fe-46fd-9070-769e9284dc88 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:07.144147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405d501-44fe-46fd-9070-769e9284dc88 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:07.144431 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405d501-44fe-46fd-9070-769e9284dc88 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:07.144841 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4810c8f-6878-4029-9fd4-76feadeadce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:07.146607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4810c8f-6878-4029-9fd4-76feadeadce7 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:07.149730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 311/624] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:07.150557 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405d501-44fe-46fd-9070-769e9284dc88 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:07.171813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-c4810c8f-6878-4029-9fd4-76feadeadce7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:07.171813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:07.171813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:07.171813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:07.171813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:07.171813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:07.171813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:07.171813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:07.171813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:07.171813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:07.171813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:07.171813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:07.171813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:07.171813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:07.171813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:07.172798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:07.172798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:07.172798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:07.172798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:07.172798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:07.172798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:07.172798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:07.172798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:07.172798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:07.172798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:07.172798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:07.172798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:07.172798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:07.172798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:07.172798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:07.172798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:07.172798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:07.173690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:07.173690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:07.173690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:07.173690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:07.176747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 314/625] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:07.179083 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c2c5b-40b6-430f-ac96-f3ddc9b01f17 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:07.179654 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c2c5b-40b6-430f-ac96-f3ddc9b01f17 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:07.180095 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c2c5b-40b6-430f-ac96-f3ddc9b01f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:07.184348 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:07.205275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-439c2c5b-40b6-430f-ac96-f3ddc9b01f17 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:07.207043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-439c2c5b-40b6-430f-ac96-f3ddc9b01f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:07.267727 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405d501-44fe-46fd-9070-769e9284dc88 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:07.271291 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:07.304868 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 315/626] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:21:07.306017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c2c5b-40b6-430f-ac96-f3ddc9b01f17 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:07.320283 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66df024-b0a7-42a0-88a6-2baf24b4f677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:07.321120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66df024-b0a7-42a0-88a6-2baf24b4f677 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:07.321503 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66df024-b0a7-42a0-88a6-2baf24b4f677 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:07.321912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66df024-b0a7-42a0-88a6-2baf24b4f677 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:07.408987 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66df024-b0a7-42a0-88a6-2baf24b4f677 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:07.410538 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66df024-b0a7-42a0-88a6-2baf24b4f677 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:07.437329 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c2c5b-40b6-430f-ac96-f3ddc9b01f17 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:07.452535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:07.477585 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3c15a-0fee-4361-b45b-eaa3a076dc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:07.478423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3c15a-0fee-4361-b45b-eaa3a076dc30 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:07.478808 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3c15a-0fee-4361-b45b-eaa3a076dc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:07.479223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3c15a-0fee-4361-b45b-eaa3a076dc30 None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:07.480176 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3c15a-0fee-4361-b45b-eaa3a076dc30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:07.480865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3c15a-0fee-4361-b45b-eaa3a076dc30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:07.482697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3c15a-0fee-4361-b45b-eaa3a076dc30 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:07.496303 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 316/627] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:07.497371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3c15a-0fee-4361-b45b-eaa3a076dc30 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:07.546014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:07.554456 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 312/628] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:21:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:21:07.556035 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66df024-b0a7-42a0-88a6-2baf24b4f677 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:07.620589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3c15a-0fee-4361-b45b-eaa3a076dc30 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:07.664796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030e91e-2b16-44cb-9346-cfbcd13365e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:07.665560 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030e91e-2b16-44cb-9346-cfbcd13365e7 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:07.665857 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030e91e-2b16-44cb-9346-cfbcd13365e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:07.666356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030e91e-2b16-44cb-9346-cfbcd13365e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:07.673103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66df024-b0a7-42a0-88a6-2baf24b4f677 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:07.678531 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030e91e-2b16-44cb-9346-cfbcd13365e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:07.686750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-e030e91e-2b16-44cb-9346-cfbcd13365e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:07.686750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:07.686750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:07.686750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:07.686750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:07.686750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:07.686750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:07.686750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:07.686750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:07.686750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:07.686750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:07.686750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:07.686750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:07.686750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:07.686750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:07.686750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:07.687773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:07.687773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:07.687773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:07.687773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:07.687773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:07.687773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:07.687773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:07.687773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:07.687773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:07.687773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:07.687773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:07.687773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:07.687773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:07.687773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:07.687773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:07.687773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:07.689223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:07.689223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:07.689223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:07.689223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:07.689223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:07.689223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:07.689223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:07.689223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:07.689223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:07.689223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:07.689223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:07.689223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:07.689223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 317/629] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:21:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:07.689223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030e91e-2b16-44cb-9346-cfbcd13365e7 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:07.710510 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:07.717937 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93b33b-22a1-40ad-98c5-32050101cb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:07.719007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93b33b-22a1-40ad-98c5-32050101cb6a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:07.719300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93b33b-22a1-40ad-98c5-32050101cb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:07.719577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93b33b-22a1-40ad-98c5-32050101cb6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:07.726511 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b93b33b-22a1-40ad-98c5-32050101cb6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:07.727993 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b93b33b-22a1-40ad-98c5-32050101cb6a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:07.742587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-0b93b33b-22a1-40ad-98c5-32050101cb6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:07.742587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:07.742587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:07.742587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:07.742587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:07.742587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:07.742587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:07.742587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:07.742587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:07.742587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:07.742587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:07.742587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:07.742587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:07.742587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:07.742587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:07.743631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:07.743631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:07.743631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:07.743631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:07.743631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:07.743631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:07.743631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:07.743631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:07.743631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:07.743631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:07.743631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:07.743631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:07.743631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:07.743631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:07.743631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:07.743631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:07.743631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:07.744699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:07.744699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:07.744699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:07.744699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:07.744699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:07.744699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:07.744699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:07.744699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:07.748444 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 313/630] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:07.749339 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93b33b-22a1-40ad-98c5-32050101cb6a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:07.797781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:07.802764 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030e91e-2b16-44cb-9346-cfbcd13365e7 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:07.838146 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbed973-b2ab-4f2d-b8f0-36f838d3157a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:07.838974 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbed973-b2ab-4f2d-b8f0-36f838d3157a None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:07.839266 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbed973-b2ab-4f2d-b8f0-36f838d3157a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:07.839546 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbed973-b2ab-4f2d-b8f0-36f838d3157a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:07.851296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbed973-b2ab-4f2d-b8f0-36f838d3157a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:07.852865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbed973-b2ab-4f2d-b8f0-36f838d3157a None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:07.862154 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93b33b-22a1-40ad-98c5-32050101cb6a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:07.874178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-6bbed973-b2ab-4f2d-b8f0-36f838d3157a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:07.874178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:07.874178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:07.874178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:07.874178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:07.874178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:07.874178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:07.874178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:07.874178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:07.874178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:07.874178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:07.874178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:07.874178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:07.874178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:07.874178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:07.875305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:07.875305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:07.875305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:07.875305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:07.875305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:07.875305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:07.875305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:07.875305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:07.875305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:07.875305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:07.875305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:07.875305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:07.875305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:07.875305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:07.875305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:07.875305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:07.875305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:07.876327 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:07.876327 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:07.876327 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:07.876327 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:07.876327 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 318/631] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:07.877050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473bacb-8364-49d3-9040-81f26021f8d8 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:07.877337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473bacb-8364-49d3-9040-81f26021f8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:07.877615 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473bacb-8364-49d3-9040-81f26021f8d8 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:07.879050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 319/632] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:07.883862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2473bacb-8364-49d3-9040-81f26021f8d8 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:07.913060 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9f3a4-cb26-45c7-8e81-5fcddbcd71db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:07.914495 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9f3a4-cb26-45c7-8e81-5fcddbcd71db None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:07.914726 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9f3a4-cb26-45c7-8e81-5fcddbcd71db None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:07.915111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9f3a4-cb26-45c7-8e81-5fcddbcd71db None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:07.915836 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9f3a4-cb26-45c7-8e81-5fcddbcd71db None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:07.916148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9f3a4-cb26-45c7-8e81-5fcddbcd71db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:07.917525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9f3a4-cb26-45c7-8e81-5fcddbcd71db None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:07.926863 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 314/633] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:21:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:07.927696 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9f3a4-cb26-45c7-8e81-5fcddbcd71db None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:07.985666 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:08.005380 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2473bacb-8364-49d3-9040-81f26021f8d8 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:08.035496 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9f3a4-cb26-45c7-8e81-5fcddbcd71db None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:08.041422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69381a4c-1668-42f7-9783-9d43be20aea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:08.042089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69381a4c-1668-42f7-9783-9d43be20aea5 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:08.042786 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69381a4c-1668-42f7-9783-9d43be20aea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:08.043104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69381a4c-1668-42f7-9783-9d43be20aea5 None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:08.043939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69381a4c-1668-42f7-9783-9d43be20aea5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:08.044608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69381a4c-1668-42f7-9783-9d43be20aea5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:08.046043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69381a4c-1668-42f7-9783-9d43be20aea5 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:08.058526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 320/634] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:08.059897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8deaa7-9fae-49c5-9d34-93b9c4396815 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:08.060223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8deaa7-9fae-49c5-9d34-93b9c4396815 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:08.060506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8deaa7-9fae-49c5-9d34-93b9c4396815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:08.068883 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6ffc6-4ba9-4bd8-b16f-20d1da4c2f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:08.069545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6ffc6-4ba9-4bd8-b16f-20d1da4c2f5c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:08.069826 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6ffc6-4ba9-4bd8-b16f-20d1da4c2f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:08.070097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6ffc6-4ba9-4bd8-b16f-20d1da4c2f5c None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:08.070809 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6ffc6-4ba9-4bd8-b16f-20d1da4c2f5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:08.071146 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6ffc6-4ba9-4bd8-b16f-20d1da4c2f5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:08.072498 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6ffc6-4ba9-4bd8-b16f-20d1da4c2f5c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:08.078013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-4f8deaa7-9fae-49c5-9d34-93b9c4396815 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:08.079234 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4f8deaa7-9fae-49c5-9d34-93b9c4396815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:08.082958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:08.085423 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 315/635] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:08.086248 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6ffc6-4ba9-4bd8-b16f-20d1da4c2f5c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:08.149371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 321/636] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:21:08.150669 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfedb26-0bd2-432e-8eae-1888cd5923b6 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:08.150994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfedb26-0bd2-432e-8eae-1888cd5923b6 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:08.151285 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfedb26-0bd2-432e-8eae-1888cd5923b6 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:08.152826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 322/637] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:08.153524 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfedb26-0bd2-432e-8eae-1888cd5923b6 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:08.185109 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6ffc6-4ba9-4bd8-b16f-20d1da4c2f5c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:08.198675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:08.221455 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2aace-f00a-4c76-bbfe-fe0f6b4e9036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:08.222243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2aace-f00a-4c76-bbfe-fe0f6b4e9036 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:08.222529 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2aace-f00a-4c76-bbfe-fe0f6b4e9036 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:08.222806 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2aace-f00a-4c76-bbfe-fe0f6b4e9036 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:08.232970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2aace-f00a-4c76-bbfe-fe0f6b4e9036 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:08.234421 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2aace-f00a-4c76-bbfe-fe0f6b4e9036 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:08.252861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-9ff2aace-f00a-4c76-bbfe-fe0f6b4e9036 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:08.252861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:08.252861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:08.252861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:08.252861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:08.252861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:08.252861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:08.252861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:08.252861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:08.252861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:08.252861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:08.252861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:08.252861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:08.252861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:08.252861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:08.253846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:08.253846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:08.253846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:08.253846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:08.253846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:08.253846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:08.253846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:08.253846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:08.253846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:08.253846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:08.253846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:08.253846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:08.253846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:08.253846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:08.253846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:08.253846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:08.253846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:08.254785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:08.254785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:08.254785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:08.254785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:08.254785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 316/638] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:08.255142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2aace-f00a-4c76-bbfe-fe0f6b4e9036 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:08.266430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfedb26-0bd2-432e-8eae-1888cd5923b6 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:08.300655 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6040574-ec27-4d45-beb2-f8f57d7fb78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:08.301549 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6040574-ec27-4d45-beb2-f8f57d7fb78d None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:08.301891 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6040574-ec27-4d45-beb2-f8f57d7fb78d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:08.302202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6040574-ec27-4d45-beb2-f8f57d7fb78d None admin] PATH_INFO: `/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/acc23233929144a5b7b3a1733da19828/roles/ea49c36b4a7f4cfb9048065638d458af` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:08.315821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:08.321240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6040574-ec27-4d45-beb2-f8f57d7fb78d None admin] RBAC: Authorizing `identity:create_grant(project_id=f33c8ac5154f46a89d1a0dc11fb58cf4, user_id=acc23233929144a5b7b3a1733da19828, role_id=ea49c36b4a7f4cfb9048065638d458af)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:08.324012 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6040574-ec27-4d45-beb2-f8f57d7fb78d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:08.340071 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 323/639] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:21:08 2022] PUT /identity/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/acc23233929144a5b7b3a1733da19828/roles/ea49c36b4a7f4cfb9048065638d458af => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:21:08.341515 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57226034-edca-4eeb-898d-500a3dbe3333 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:08.342136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57226034-edca-4eeb-898d-500a3dbe3333 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:08.342136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57226034-edca-4eeb-898d-500a3dbe3333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:08.361081 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-57226034-edca-4eeb-898d-500a3dbe3333 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:08.362320 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-57226034-edca-4eeb-898d-500a3dbe3333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:08.367726 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2aace-f00a-4c76-bbfe-fe0f6b4e9036 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:08.412500 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b3330-dcde-4e0f-983d-d4acc244135b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:08.413345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b3330-dcde-4e0f-983d-d4acc244135b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:08.413633 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b3330-dcde-4e0f-983d-d4acc244135b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:08.413911 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b3330-dcde-4e0f-983d-d4acc244135b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:08.421583 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b3330-dcde-4e0f-983d-d4acc244135b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:08.423104 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b3330-dcde-4e0f-983d-d4acc244135b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:08.438910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-7e8b3330-dcde-4e0f-983d-d4acc244135b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:08.438910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:08.438910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:08.438910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:08.438910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:08.438910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:08.438910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:08.438910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:08.438910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:08.438910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:08.438910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:08.438910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:08.438910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:08.438910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:08.438910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:08.439881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:08.439881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:08.439881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:08.439881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:08.439881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:08.439881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:08.439881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:08.439881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:08.439881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:08.439881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:08.439881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:08.439881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:08.439881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:08.439881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:08.439881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:08.439881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:08.439881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:08.440789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:08.440789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:08.440789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:08.440789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:08.440789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:08.440789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:08.440789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:08.440789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:08.445179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 317/640] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:08.446187 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b3330-dcde-4e0f-983d-d4acc244135b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:08.451291 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 324/641] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:21:08.451291 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57226034-edca-4eeb-898d-500a3dbe3333 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:08.495885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:08.585916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b3330-dcde-4e0f-983d-d4acc244135b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:08.586154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57226034-edca-4eeb-898d-500a3dbe3333 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:08.607947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:08.624714 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb7b29-0709-4d6c-9a09-b9a880056c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:08.625705 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb7b29-0709-4d6c-9a09-b9a880056c6e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:08.626292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb7b29-0709-4d6c-9a09-b9a880056c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:08.626739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb7b29-0709-4d6c-9a09-b9a880056c6e None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:08.627550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb7b29-0709-4d6c-9a09-b9a880056c6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:08.627861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb7b29-0709-4d6c-9a09-b9a880056c6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:08.629251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb7b29-0709-4d6c-9a09-b9a880056c6e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:08.631769 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a1052-eb0b-4bfe-b22e-05dc7655f29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:08.632146 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a1052-eb0b-4bfe-b22e-05dc7655f29b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:08.632441 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a1052-eb0b-4bfe-b22e-05dc7655f29b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:08.632747 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a1052-eb0b-4bfe-b22e-05dc7655f29b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:08.645203 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a1052-eb0b-4bfe-b22e-05dc7655f29b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:08.647710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 325/642] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:08.648356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb7b29-0709-4d6c-9a09-b9a880056c6e None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:08.656877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-093a1052-eb0b-4bfe-b22e-05dc7655f29b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:08.656877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:08.656877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:08.656877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:08.656877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:08.656877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:08.656877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:08.656877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:08.656877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:08.656877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:08.656877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:08.656877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:08.656877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:08.656877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:08.656877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:08.656877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:08.658007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:08.658007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:08.658007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:08.658007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:08.658007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:08.658007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:08.658007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:08.658007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:08.658007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:08.658007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:08.658007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:08.658007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:08.658007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:08.658007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:08.658007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:08.658007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:08.658906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:08.658906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:08.658906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:08.658906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:08.658906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:08.658906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:08.658906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:08.658906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:08.658906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:08.658906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:08.658906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:08.658906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:08.658906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 318/643] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:21:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:08.659653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a1052-eb0b-4bfe-b22e-05dc7655f29b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:08.760976 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:08.761727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb7b29-0709-4d6c-9a09-b9a880056c6e None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:08.784669 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a1052-eb0b-4bfe-b22e-05dc7655f29b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:08.799268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b03cd-7d69-4871-bf8b-3cac23d0dd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:08.799940 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b03cd-7d69-4871-bf8b-3cac23d0dd85 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:08.800229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b03cd-7d69-4871-bf8b-3cac23d0dd85 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:08.800508 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b03cd-7d69-4871-bf8b-3cac23d0dd85 None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:08.801312 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b03cd-7d69-4871-bf8b-3cac23d0dd85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:08.801625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b03cd-7d69-4871-bf8b-3cac23d0dd85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:08.803103 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b03cd-7d69-4871-bf8b-3cac23d0dd85 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:08.828013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 326/644] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:08.829902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b03cd-7d69-4871-bf8b-3cac23d0dd85 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:08.841994 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05ddf9-aed8-4105-acdc-8abb3640ddc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:08.842805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05ddf9-aed8-4105-acdc-8abb3640ddc4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:08.843128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05ddf9-aed8-4105-acdc-8abb3640ddc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:08.843408 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05ddf9-aed8-4105-acdc-8abb3640ddc4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:08.854822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05ddf9-aed8-4105-acdc-8abb3640ddc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:08.863207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-2f05ddf9-aed8-4105-acdc-8abb3640ddc4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:08.863207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:08.863207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:08.863207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:08.863207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:08.863207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:08.863207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:08.863207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:08.863207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:08.863207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:08.863207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:08.863207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:08.863207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:08.863207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:08.863207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:08.863207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:08.864492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:08.864492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:08.864492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:08.864492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:08.864492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:08.864492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:08.864492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:08.864492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:08.864492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:08.864492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:08.864492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:08.864492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:08.864492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:08.864492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:08.864492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:08.864492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:08.865555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:08.865555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:08.865555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:08.865555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:08.865555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:08.865555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:08.865555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:08.865555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:08.865555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:08.865555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:08.865555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:08.865555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:08.865555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 319/645] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:21:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:08.869900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05ddf9-aed8-4105-acdc-8abb3640ddc4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:08.911233 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:08.983152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05ddf9-aed8-4105-acdc-8abb3640ddc4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:08.989151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b03cd-7d69-4871-bf8b-3cac23d0dd85 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:09.026519 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622153f-cb79-4149-b979-542d46901d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:09.027368 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622153f-cb79-4149-b979-542d46901d1c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:09.027756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622153f-cb79-4149-b979-542d46901d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:09.028145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622153f-cb79-4149-b979-542d46901d1c None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:09.028284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434400c-62b9-44f0-92a6-05407f427826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:09.028979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434400c-62b9-44f0-92a6-05407f427826 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:09.029248 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622153f-cb79-4149-b979-542d46901d1c None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:09.029407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434400c-62b9-44f0-92a6-05407f427826 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:09.029558 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434400c-62b9-44f0-92a6-05407f427826 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:09.029697 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622153f-cb79-4149-b979-542d46901d1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:09.031452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622153f-cb79-4149-b979-542d46901d1c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:09.039165 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434400c-62b9-44f0-92a6-05407f427826 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:09.040660 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434400c-62b9-44f0-92a6-05407f427826 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:09.042357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 320/646] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:21:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:09.048412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622153f-cb79-4149-b979-542d46901d1c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:09.059066 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:09.059603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-e434400c-62b9-44f0-92a6-05407f427826 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 20:21:09.059603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:09.059603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:09.059603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:09.059603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:09.059603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:09.059603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:09.059603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:09.059603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:09.059603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:09.059603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:09.059603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:09.059603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:09.059603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:09.059603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:09.059603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:09.061373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:09.061373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:09.061373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:09.061373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:09.061373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:09.061373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:09.061373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:09.061373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:09.061373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:09.061373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:09.061373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:09.061373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:09.061373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:09.061373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:09.061373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:09.061373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:09.061373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:09.061373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:09.062755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:09.062755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:09.062755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:09.062755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:09.062755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:09.062755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:09.062755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:09.062755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:09.062755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 20:21:09.062755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:09.062755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 327/647] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:21:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:09.062755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434400c-62b9-44f0-92a6-05407f427826 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:09.175674 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622153f-cb79-4149-b979-542d46901d1c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:09.179445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:09.182562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434400c-62b9-44f0-92a6-05407f427826 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:09.213816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c21ff-0670-414f-b722-0fb5e5fdf55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:09.214484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c21ff-0670-414f-b722-0fb5e5fdf55f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:09.214771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c21ff-0670-414f-b722-0fb5e5fdf55f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:09.215098 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c21ff-0670-414f-b722-0fb5e5fdf55f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:09.215925 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c21ff-0670-414f-b722-0fb5e5fdf55f None admin] RBAC: Adding query filter params (role.id=ea49c36b4a7f4cfb9048065638d458af, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=aa3bb9f1e24a4e9ebd460e97b3736bc0) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:09.216776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c21ff-0670-414f-b722-0fb5e5fdf55f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:09.218217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c21ff-0670-414f-b722-0fb5e5fdf55f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:09.220883 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a527688-4346-4021-908c-c1c60429f5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:09.221556 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a527688-4346-4021-908c-c1c60429f5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:09.221844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a527688-4346-4021-908c-c1c60429f5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:09.222122 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a527688-4346-4021-908c-c1c60429f5e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:09.222874 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a527688-4346-4021-908c-c1c60429f5e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:09.223231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a527688-4346-4021-908c-c1c60429f5e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:09.235310 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a527688-4346-4021-908c-c1c60429f5e6 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:09.235524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 321/648] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:21:09 2022] GET /identity/v3/role_assignments?user.id=aa3bb9f1e24a4e9ebd460e97b3736bc0&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=ea49c36b4a7f4cfb9048065638d458af => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:09.235524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c21ff-0670-414f-b722-0fb5e5fdf55f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:09.244826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 328/649] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:21:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:09.245581 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a527688-4346-4021-908c-c1c60429f5e6 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:09.326577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:09.365402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c21ff-0670-414f-b722-0fb5e5fdf55f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:09.390711 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a527688-4346-4021-908c-c1c60429f5e6 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:09.406627 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bc495-cfd1-47cc-a9a2-f7b3f72204f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:09.407662 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bc495-cfd1-47cc-a9a2-f7b3f72204f5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:09.408049 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bc495-cfd1-47cc-a9a2-f7b3f72204f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:09.408337 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bc495-cfd1-47cc-a9a2-f7b3f72204f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:09.428503 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bc495-cfd1-47cc-a9a2-f7b3f72204f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:09.429980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bc495-cfd1-47cc-a9a2-f7b3f72204f5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:09.432062 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa677f6c-23bf-4409-b19f-8c30bb3829fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:09.432747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa677f6c-23bf-4409-b19f-8c30bb3829fe None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:09.433086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa677f6c-23bf-4409-b19f-8c30bb3829fe None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:09.433798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa677f6c-23bf-4409-b19f-8c30bb3829fe None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:09.434657 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa677f6c-23bf-4409-b19f-8c30bb3829fe None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:09.435026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa677f6c-23bf-4409-b19f-8c30bb3829fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:09.436448 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa677f6c-23bf-4409-b19f-8c30bb3829fe None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:09.454290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-723bc495-cfd1-47cc-a9a2-f7b3f72204f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:09.454290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:09.454290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:09.454290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:09.454290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:09.454290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:09.454290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:09.454290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:09.454290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:09.454290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:09.454290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:09.454290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:09.454290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:09.454290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:09.454290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:09.455290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:09.455290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:09.455290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:09.455290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:09.455290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:09.455290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:09.455290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:09.455290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:09.455290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:09.455290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:09.455290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:09.455290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:09.455290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:09.455290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:09.455290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:09.455290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:09.455290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:09.456162 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:09.456162 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:09.456162 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:09.456162 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:09.456162 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 322/650] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:09.456934 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bc495-cfd1-47cc-a9a2-f7b3f72204f5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:09.463125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 329/651] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jul 14 20:21:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:09.463125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa677f6c-23bf-4409-b19f-8c30bb3829fe None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:09.492194 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:09.592143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa677f6c-23bf-4409-b19f-8c30bb3829fe None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:09.595091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bc495-cfd1-47cc-a9a2-f7b3f72204f5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:09.634781 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:09.641770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0c0f6-5095-4676-9b25-b649f94782b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:09.642160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d682ebe-9637-44be-aca4-73e28cb6c11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:09.642712 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0c0f6-5095-4676-9b25-b649f94782b7 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:09.642840 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d682ebe-9637-44be-aca4-73e28cb6c11c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:09.643132 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d682ebe-9637-44be-aca4-73e28cb6c11c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:09.643270 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0c0f6-5095-4676-9b25-b649f94782b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:09.643407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d682ebe-9637-44be-aca4-73e28cb6c11c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:09.643767 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0c0f6-5095-4676-9b25-b649f94782b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:09.645095 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0c0f6-5095-4676-9b25-b649f94782b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:09.645556 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0c0f6-5095-4676-9b25-b649f94782b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:09.647046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0c0f6-5095-4676-9b25-b649f94782b7 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:09.657321 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d682ebe-9637-44be-aca4-73e28cb6c11c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:09.658890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d682ebe-9637-44be-aca4-73e28cb6c11c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:09.660193 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 330/652] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:09.661099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0c0f6-5095-4676-9b25-b649f94782b7 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:09.677933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-6d682ebe-9637-44be-aca4-73e28cb6c11c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:09.677933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:09.677933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:09.677933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:09.677933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:09.677933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:09.677933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:09.677933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:09.677933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:09.677933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:09.677933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:09.677933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:09.677933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:09.677933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:09.677933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:09.679068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:09.679068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:09.679068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:09.679068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:09.679068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:09.679068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:09.679068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:09.679068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:09.679068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:09.679068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:09.679068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:09.679068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:09.679068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:09.679068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:09.679068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:09.679068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:09.679068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:09.680223 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:09.680223 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:09.680223 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:09.680223 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:09.680223 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 323/653] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:09.681422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d682ebe-9637-44be-aca4-73e28cb6c11c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:09.750733 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:09.772001 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0c0f6-5095-4676-9b25-b649f94782b7 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:09.793342 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d682ebe-9637-44be-aca4-73e28cb6c11c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:09.812515 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9448cf9-7f6c-4f63-9286-9196cad50f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:09.813387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9448cf9-7f6c-4f63-9286-9196cad50f04 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:09.813795 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9448cf9-7f6c-4f63-9286-9196cad50f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:09.814145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9448cf9-7f6c-4f63-9286-9196cad50f04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:09.833552 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad3076-1335-43a2-8134-2c7f10fc42b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:09.834783 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad3076-1335-43a2-8134-2c7f10fc42b3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:09.835304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad3076-1335-43a2-8134-2c7f10fc42b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:09.835796 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad3076-1335-43a2-8134-2c7f10fc42b3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:09.844523 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad3076-1335-43a2-8134-2c7f10fc42b3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:09.846393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad3076-1335-43a2-8134-2c7f10fc42b3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:09.862069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-d7ad3076-1335-43a2-8134-2c7f10fc42b3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 20:21:09.862069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:09.862069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:09.862069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:09.862069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:09.862069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:09.862069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:09.862069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:09.862069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:09.862069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:09.862069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:09.862069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:09.862069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:09.862069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:09.862069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:09.862069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:09.863133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:09.863133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:09.863133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:09.863133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:09.863133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:09.863133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:09.863133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:09.863133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:09.863133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:09.863133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:09.863133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:09.863133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:09.863133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:09.863133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:09.863133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:09.863133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:09.863133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:09.863133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:09.864088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:09.864088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:09.864088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:09.864088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:09.864088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:09.864088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:09.864088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:09.864088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:09.864088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 20:21:09.864088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:09.866895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 324/654] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:21:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:09.868079 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad3076-1335-43a2-8134-2c7f10fc42b3 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:09.888774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9448cf9-7f6c-4f63-9286-9196cad50f04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:09.890474 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9448cf9-7f6c-4f63-9286-9196cad50f04 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:09.908712 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:09.988866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad3076-1335-43a2-8134-2c7f10fc42b3 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:09.994645 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:10.029917 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 331/655] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:21:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:21:10.029917 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9448cf9-7f6c-4f63-9286-9196cad50f04 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:10.030727 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24f16d-2fa6-4d7b-9d74-e5210639bfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:10.031437 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24f16d-2fa6-4d7b-9d74-e5210639bfb3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:10.031711 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24f16d-2fa6-4d7b-9d74-e5210639bfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:10.031989 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24f16d-2fa6-4d7b-9d74-e5210639bfb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:10.032825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24f16d-2fa6-4d7b-9d74-e5210639bfb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:10.033962 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24f16d-2fa6-4d7b-9d74-e5210639bfb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:10.035236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24f16d-2fa6-4d7b-9d74-e5210639bfb3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:10.049812 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 325/656] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:10.050569 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24f16d-2fa6-4d7b-9d74-e5210639bfb3 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:10.151985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9448cf9-7f6c-4f63-9286-9196cad50f04 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:10.155353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:10.164287 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24f16d-2fa6-4d7b-9d74-e5210639bfb3 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:10.198032 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b12f72-3817-4706-a07b-f67d26d3a7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:10.198737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b12f72-3817-4706-a07b-f67d26d3a7de None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:10.199049 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b12f72-3817-4706-a07b-f67d26d3a7de None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:10.199325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b12f72-3817-4706-a07b-f67d26d3a7de None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:10.200162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b12f72-3817-4706-a07b-f67d26d3a7de None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:10.200472 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b12f72-3817-4706-a07b-f67d26d3a7de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:10.202321 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b12f72-3817-4706-a07b-f67d26d3a7de None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:10.204889 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7df48e-9648-4ba5-bbd4-65b01241b354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:10.205939 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7df48e-9648-4ba5-bbd4-65b01241b354 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:10.206444 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7df48e-9648-4ba5-bbd4-65b01241b354 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:10.206911 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7df48e-9648-4ba5-bbd4-65b01241b354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:10.218191 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7df48e-9648-4ba5-bbd4-65b01241b354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:10.220022 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7df48e-9648-4ba5-bbd4-65b01241b354 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:10.222156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 332/657] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jul 14 20:21:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:10.223103 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b12f72-3817-4706-a07b-f67d26d3a7de None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:10.240030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-da7df48e-9648-4ba5-bbd4-65b01241b354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:10.240030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:10.240030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:10.240030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:10.240030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:10.240030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:10.240030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:10.240030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:10.240030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:10.240030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:10.240030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:10.240030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:10.240030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:10.240030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:10.240030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:10.241051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:10.241051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:10.241051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:10.241051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:10.241051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:10.241051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:10.241051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:10.241051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:10.241051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:10.241051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:10.241051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:10.241051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:10.241051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:10.241051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:10.241051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:10.241051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:10.241051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:10.241914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:10.241914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:10.241914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:10.241914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:10.249174 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 326/658] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:10.250368 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7df48e-9648-4ba5-bbd4-65b01241b354 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:10.265405 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:10.330835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b12f72-3817-4706-a07b-f67d26d3a7de None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:10.367461 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9027127-c439-4fbd-808c-afb91b1e723c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:10.368262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9027127-c439-4fbd-808c-afb91b1e723c None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:10.368545 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9027127-c439-4fbd-808c-afb91b1e723c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:10.368817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9027127-c439-4fbd-808c-afb91b1e723c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:10.370672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7df48e-9648-4ba5-bbd4-65b01241b354 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:10.377208 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9027127-c439-4fbd-808c-afb91b1e723c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:10.378646 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9027127-c439-4fbd-808c-afb91b1e723c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:10.394190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-e9027127-c439-4fbd-808c-afb91b1e723c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 20:21:10.394190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:10.394190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:10.394190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:10.394190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:10.394190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:10.394190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:10.394190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:10.394190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:10.394190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:10.394190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:10.394190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:10.394190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:10.394190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:10.394190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:10.394190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:10.395227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:10.395227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:10.395227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:10.395227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:10.395227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:10.395227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:10.395227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:10.395227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:10.395227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:10.395227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:10.395227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:10.395227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:10.395227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:10.395227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:10.395227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:10.395227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:10.395227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:10.395227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:10.396170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:10.396170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:10.396170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:10.396170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:10.396170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:10.396170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:10.396170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:10.396170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:10.396170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 20:21:10.396170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:10.396170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 333/659] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:21:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:10.396767 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9027127-c439-4fbd-808c-afb91b1e723c None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:10.413363 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c0fb6-3728-40d9-b907-9eb159e74c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:10.414387 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c0fb6-3728-40d9-b907-9eb159e74c4e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:10.414869 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c0fb6-3728-40d9-b907-9eb159e74c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:10.415452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c0fb6-3728-40d9-b907-9eb159e74c4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:10.432407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:10.489489 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c0fb6-3728-40d9-b907-9eb159e74c4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:10.490973 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c0fb6-3728-40d9-b907-9eb159e74c4e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:10.503394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:10.510968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9027127-c439-4fbd-808c-afb91b1e723c None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:10.552899 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb2d4b-824f-46b7-b74f-314f5cac953c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:10.553592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb2d4b-824f-46b7-b74f-314f5cac953c None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:10.553881 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb2d4b-824f-46b7-b74f-314f5cac953c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:10.554161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb2d4b-824f-46b7-b74f-314f5cac953c None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:10.554906 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb2d4b-824f-46b7-b74f-314f5cac953c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:10.555263 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb2d4b-824f-46b7-b74f-314f5cac953c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:10.556668 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb2d4b-824f-46b7-b74f-314f5cac953c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:10.569899 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 334/660] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:10.570719 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb2d4b-824f-46b7-b74f-314f5cac953c None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:10.610243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 327/661] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:21:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:21:10.611633 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37368358-56f9-4b76-94e1-52c23a17efed None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:10.611922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37368358-56f9-4b76-94e1-52c23a17efed None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:10.612199 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37368358-56f9-4b76-94e1-52c23a17efed None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:10.615058 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 328/662] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:10.615058 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37368358-56f9-4b76-94e1-52c23a17efed None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:10.671419 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:10.693210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb2d4b-824f-46b7-b74f-314f5cac953c None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:10.728384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f13c65-3c91-4a6d-8d20-7df253dcd285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:10.729407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f13c65-3c91-4a6d-8d20-7df253dcd285 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:10.729698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f13c65-3c91-4a6d-8d20-7df253dcd285 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:10.729977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f13c65-3c91-4a6d-8d20-7df253dcd285 None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:10.730814 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f13c65-3c91-4a6d-8d20-7df253dcd285 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:10.731159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f13c65-3c91-4a6d-8d20-7df253dcd285 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:10.731287 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37368358-56f9-4b76-94e1-52c23a17efed None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:10.733146 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f13c65-3c91-4a6d-8d20-7df253dcd285 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:10.754690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 335/663] 10.222.0.52 () {56 vars in 1139 bytes} [Thu Jul 14 20:21:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:10.754690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f13c65-3c91-4a6d-8d20-7df253dcd285 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:10.770533 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f48e8-729e-492f-bf29-6f80462c40e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:10.771370 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f48e8-729e-492f-bf29-6f80462c40e9 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:10.771654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f48e8-729e-492f-bf29-6f80462c40e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:10.772210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:10.772316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f48e8-729e-492f-bf29-6f80462c40e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:10.780640 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f48e8-729e-492f-bf29-6f80462c40e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:10.782089 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f48e8-729e-492f-bf29-6f80462c40e9 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:10.797952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-df9f48e8-729e-492f-bf29-6f80462c40e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:10.797952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:10.797952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:10.797952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:10.797952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:10.797952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:10.797952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:10.797952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:10.797952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:10.797952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:10.797952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:10.797952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:10.797952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:10.797952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:10.797952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:10.798862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:10.798862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:10.798862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:10.798862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:10.798862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:10.798862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:10.798862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:10.798862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:10.798862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:10.798862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:10.798862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:10.798862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:10.798862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:10.798862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:10.798862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:10.798862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:10.798862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:10.799791 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:10.799791 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:10.799791 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:10.799791 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:10.799791 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:10.799791 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:10.799791 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:10.799791 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:10.803121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 329/664] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:10.804378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d58e9b-f1dc-4171-b5a2-8d6507991cba None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:10.804669 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d58e9b-f1dc-4171-b5a2-8d6507991cba None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:10.804956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d58e9b-f1dc-4171-b5a2-8d6507991cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:10.823246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c1d58e9b-f1dc-4171-b5a2-8d6507991cba None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:10.824660 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c1d58e9b-f1dc-4171-b5a2-8d6507991cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:10.868871 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f13c65-3c91-4a6d-8d20-7df253dcd285 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:10.898980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 330/665] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:21:10.898980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d58e9b-f1dc-4171-b5a2-8d6507991cba None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:10.910761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366899d2-956a-428e-890a-9d1c34798c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:10.911810 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366899d2-956a-428e-890a-9d1c34798c7d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:10.912239 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366899d2-956a-428e-890a-9d1c34798c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:10.912657 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366899d2-956a-428e-890a-9d1c34798c7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:10.924682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366899d2-956a-428e-890a-9d1c34798c7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:10.926072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366899d2-956a-428e-890a-9d1c34798c7d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:10.944053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-366899d2-956a-428e-890a-9d1c34798c7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:10.944053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:10.944053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:10.944053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:10.944053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:10.944053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:10.944053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:10.944053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:10.944053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:10.944053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:10.944053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:10.944053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:10.944053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:10.944053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:10.944053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:10.945148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:10.945148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:10.945148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:10.945148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:10.945148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:10.945148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:10.945148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:10.945148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:10.945148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:10.945148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:10.945148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:10.945148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:10.945148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:10.945148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:10.945148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:10.945148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:10.945148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:10.946072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:10.946072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:10.946072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:10.946072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:10.946072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 336/666] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:10.946072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366899d2-956a-428e-890a-9d1c34798c7d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:10.952634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:11.023143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:11.028211 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d58e9b-f1dc-4171-b5a2-8d6507991cba None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:11.068180 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366899d2-956a-428e-890a-9d1c34798c7d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:11.071251 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a554ca-1e61-4966-bb18-816e30e113fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:11.071423 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a554ca-1e61-4966-bb18-816e30e113fe None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:11.071635 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a554ca-1e61-4966-bb18-816e30e113fe None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:11.071835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a554ca-1e61-4966-bb18-816e30e113fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:11.106280 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf82cd7-ebdc-47b2-b540-573cca1f79c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:11.107266 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf82cd7-ebdc-47b2-b540-573cca1f79c2 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:11.107729 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf82cd7-ebdc-47b2-b540-573cca1f79c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:11.108009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf82cd7-ebdc-47b2-b540-573cca1f79c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:11.108792 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf82cd7-ebdc-47b2-b540-573cca1f79c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:11.109102 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf82cd7-ebdc-47b2-b540-573cca1f79c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:11.110542 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf82cd7-ebdc-47b2-b540-573cca1f79c2 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:11.127484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 337/667] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:11.127484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf82cd7-ebdc-47b2-b540-573cca1f79c2 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:11.146429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a554ca-1e61-4966-bb18-816e30e113fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:11.147815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a554ca-1e61-4966-bb18-816e30e113fe None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:11.185819 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:11.235225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf82cd7-ebdc-47b2-b540-573cca1f79c2 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:11.256633 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 331/668] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:21:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 20:21:11.256633 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a554ca-1e61-4966-bb18-816e30e113fe None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:11.267826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:11.276483 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bf897-3d4f-4192-a7d8-1602898289c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:11.277279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bf897-3d4f-4192-a7d8-1602898289c0 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:11.277559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bf897-3d4f-4192-a7d8-1602898289c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:11.277836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bf897-3d4f-4192-a7d8-1602898289c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:11.289634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5bf897-3d4f-4192-a7d8-1602898289c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:11.297579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-3a5bf897-3d4f-4192-a7d8-1602898289c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:11.297579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:11.297579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:11.297579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:11.297579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:11.297579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:11.297579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:11.297579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:11.297579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:11.297579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:11.297579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:11.297579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:11.297579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:11.297579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:11.297579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:11.297579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:11.298707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:11.298707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:11.298707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:11.298707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:11.298707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:11.298707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:11.298707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:11.298707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:11.298707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:11.298707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:11.298707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:11.298707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:11.298707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:11.298707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:11.298707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:11.298707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:11.299615 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:11.299615 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:11.299615 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:11.299615 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:11.299615 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:11.299615 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:11.299615 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:11.299615 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:11.299615 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:11.299615 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:11.299615 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:11.299615 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:11.299615 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 338/669] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:21:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:11.300312 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bf897-3d4f-4192-a7d8-1602898289c0 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:11.375084 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a554ca-1e61-4966-bb18-816e30e113fe None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:11.406883 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bf897-3d4f-4192-a7d8-1602898289c0 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:11.413758 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7db51-d604-423e-a866-82bd47d5c3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:11.414822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7db51-d604-423e-a866-82bd47d5c3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:11.415208 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7db51-d604-423e-a866-82bd47d5c3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:11.415425 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7db51-d604-423e-a866-82bd47d5c3b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:11.416151 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7db51-d604-423e-a866-82bd47d5c3b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:11.416815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7db51-d604-423e-a866-82bd47d5c3b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:11.418600 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7db51-d604-423e-a866-82bd47d5c3b6 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:11.420986 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:11.436430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 332/670] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:11.436430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7db51-d604-423e-a866-82bd47d5c3b6 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:11.451805 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d3c41e-3d30-4725-ad97-2c092b39866d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:11.452467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3c41e-3d30-4725-ad97-2c092b39866d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:11.452751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3c41e-3d30-4725-ad97-2c092b39866d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:11.453025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3c41e-3d30-4725-ad97-2c092b39866d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:11.453858 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d3c41e-3d30-4725-ad97-2c092b39866d None admin] RBAC: Adding query filter params (role.id=ea49c36b4a7f4cfb9048065638d458af, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=4b7a38b4a7754037a00d11dafc88645b) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:11.454170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d3c41e-3d30-4725-ad97-2c092b39866d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:11.455719 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d3c41e-3d30-4725-ad97-2c092b39866d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:11.461660 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 339/671] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:21:11 2022] GET /identity/v3/role_assignments?user.id=4b7a38b4a7754037a00d11dafc88645b&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=ea49c36b4a7f4cfb9048065638d458af => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:11.462435 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d3c41e-3d30-4725-ad97-2c092b39866d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:11.531255 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:11.546522 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7db51-d604-423e-a866-82bd47d5c3b6 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:11.589964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ed1b0-133a-4a80-9cf3-2f75af2a58db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:11.590923 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ed1b0-133a-4a80-9cf3-2f75af2a58db None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:11.591065 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ed1b0-133a-4a80-9cf3-2f75af2a58db None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:11.591377 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ed1b0-133a-4a80-9cf3-2f75af2a58db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:11.594442 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d3c41e-3d30-4725-ad97-2c092b39866d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:11.603749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ed1b0-133a-4a80-9cf3-2f75af2a58db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:11.605237 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ed1b0-133a-4a80-9cf3-2f75af2a58db None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:11.626275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-361ed1b0-133a-4a80-9cf3-2f75af2a58db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:11.626275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:11.626275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:11.626275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:11.626275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:11.626275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:11.626275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:11.626275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:11.626275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:11.626275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:11.626275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:11.626275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:11.626275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:11.626275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:11.626275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:11.627401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:11.627401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:11.627401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:11.627401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:11.627401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:11.627401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:11.627401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:11.627401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:11.627401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:11.627401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:11.627401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:11.627401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:11.627401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:11.627401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:11.627401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:11.627401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:11.627401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:11.628367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:11.628367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:11.628367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:11.628367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:11.630894 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 333/672] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:11.635150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ed1b0-133a-4a80-9cf3-2f75af2a58db None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:11.645215 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddad4f-15f7-4578-8afe-806f00766164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:11.645918 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddad4f-15f7-4578-8afe-806f00766164 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:11.646205 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddad4f-15f7-4578-8afe-806f00766164 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:11.646485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddad4f-15f7-4578-8afe-806f00766164 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:11.647397 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ddad4f-15f7-4578-8afe-806f00766164 None admin] RBAC: Adding query filter params (name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:11.647709 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ddad4f-15f7-4578-8afe-806f00766164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:11.649184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ddad4f-15f7-4578-8afe-806f00766164 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:11.658975 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 340/673] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:21:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:11.661004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df78d5-a835-4730-ae5b-754c397d7160 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:11.661432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df78d5-a835-4730-ae5b-754c397d7160 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:11.661828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df78d5-a835-4730-ae5b-754c397d7160 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:11.663575 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 341/674] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:11.664568 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df78d5-a835-4730-ae5b-754c397d7160 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:11.700028 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:11.759323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ed1b0-133a-4a80-9cf3-2f75af2a58db None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:11.782012 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df78d5-a835-4730-ae5b-754c397d7160 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:11.795747 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2c46f-60ca-4c9b-b927-92b57c0fde0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:11.796506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:11.796618 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2c46f-60ca-4c9b-b927-92b57c0fde0f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:11.796822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2c46f-60ca-4c9b-b927-92b57c0fde0f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:11.797087 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2c46f-60ca-4c9b-b927-92b57c0fde0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:11.805148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2c46f-60ca-4c9b-b927-92b57c0fde0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:11.806591 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2c46f-60ca-4c9b-b927-92b57c0fde0f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:11.820328 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766ecb5-8a54-4bc5-96d5-09e08ff8d0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:11.821001 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766ecb5-8a54-4bc5-96d5-09e08ff8d0fc None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:11.821285 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766ecb5-8a54-4bc5-96d5-09e08ff8d0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:11.821566 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766ecb5-8a54-4bc5-96d5-09e08ff8d0fc None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:11.821675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-0ca2c46f-60ca-4c9b-b927-92b57c0fde0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:11.821675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:11.821675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:11.821675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:11.821675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:11.821675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:11.821675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:11.821675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:11.821675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:11.821675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:11.821675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:11.821675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:11.821675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:11.821675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:11.821675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:11.822493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:11.822493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:11.822493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:11.822493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:11.822493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:11.822493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:11.822493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:11.822493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:11.822493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:11.822493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:11.822493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:11.822493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:11.822493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:11.822493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:11.822493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:11.822493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:11.822493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:11.823432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:11.823432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:11.823432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:11.823432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:11.823432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:11.823432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:11.823432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:11.823432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:11.823864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766ecb5-8a54-4bc5-96d5-09e08ff8d0fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:11.823864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766ecb5-8a54-4bc5-96d5-09e08ff8d0fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:11.823996 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766ecb5-8a54-4bc5-96d5-09e08ff8d0fc None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:11.827597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 334/675] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:11.828429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2c46f-60ca-4c9b-b927-92b57c0fde0f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:11.842311 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 342/676] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:11.842311 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d0f1f-97cb-4692-93f1-072c7eb26b49 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:11.842311 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d0f1f-97cb-4692-93f1-072c7eb26b49 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:11.842311 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d0f1f-97cb-4692-93f1-072c7eb26b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:11.858469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-a26d0f1f-97cb-4692-93f1-072c7eb26b49 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:11.859715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-a26d0f1f-97cb-4692-93f1-072c7eb26b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:11.935508 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 343/677] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:21:11.935508 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d0f1f-97cb-4692-93f1-072c7eb26b49 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:11.952844 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2c46f-60ca-4c9b-b927-92b57c0fde0f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:11.962954 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:11.988790 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc23744-b85b-40bc-803f-b19a9a212729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:11.989591 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc23744-b85b-40bc-803f-b19a9a212729 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:11.989874 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc23744-b85b-40bc-803f-b19a9a212729 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:11.990149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc23744-b85b-40bc-803f-b19a9a212729 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:12.001052 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc23744-b85b-40bc-803f-b19a9a212729 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:12.002511 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc23744-b85b-40bc-803f-b19a9a212729 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:12.022584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-9bc23744-b85b-40bc-803f-b19a9a212729 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:12.022584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:12.022584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:12.022584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:12.022584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:12.022584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:12.022584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:12.022584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:12.022584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:12.022584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:12.022584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:12.022584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:12.022584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:12.022584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:12.022584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:12.023638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:12.023638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:12.023638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:12.023638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:12.023638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:12.023638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:12.023638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:12.023638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:12.023638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:12.023638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:12.023638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:12.023638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:12.023638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:12.023638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:12.023638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:12.023638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:12.023638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:12.029772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:12.029772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:12.029772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:12.029772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:12.029772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 335/678] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:12.029772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc23744-b85b-40bc-803f-b19a9a212729 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:12.049072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d0f1f-97cb-4692-93f1-072c7eb26b49 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:12.051520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:12.086053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783309c9-b621-4443-99da-1434299d8a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:12.086722 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783309c9-b621-4443-99da-1434299d8a3c None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:12.087037 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783309c9-b621-4443-99da-1434299d8a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:12.087316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783309c9-b621-4443-99da-1434299d8a3c None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:12.088109 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783309c9-b621-4443-99da-1434299d8a3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:12.088415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783309c9-b621-4443-99da-1434299d8a3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:12.089845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783309c9-b621-4443-99da-1434299d8a3c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:12.102276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 344/679] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:12.103123 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783309c9-b621-4443-99da-1434299d8a3c None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:12.147844 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc23744-b85b-40bc-803f-b19a9a212729 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:12.185512 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabbc56-a4ba-43a2-b7e7-224574d13946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:12.186616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabbc56-a4ba-43a2-b7e7-224574d13946 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:12.186899 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabbc56-a4ba-43a2-b7e7-224574d13946 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:12.187194 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabbc56-a4ba-43a2-b7e7-224574d13946 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:12.194699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabbc56-a4ba-43a2-b7e7-224574d13946 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:12.196224 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabbc56-a4ba-43a2-b7e7-224574d13946 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:12.208664 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783309c9-b621-4443-99da-1434299d8a3c None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:12.210900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-8fabbc56-a4ba-43a2-b7e7-224574d13946 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:12.210900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:12.210900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:12.210900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:12.210900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:12.210900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:12.210900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:12.210900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:12.210900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:12.210900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:12.210900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:12.210900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:12.210900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:12.210900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:12.210900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:12.211837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:12.211837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:12.211837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:12.211837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:12.211837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:12.211837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:12.211837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:12.211837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:12.211837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:12.211837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:12.211837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:12.211837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:12.211837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:12.211837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:12.211837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:12.211837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:12.211837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:12.212707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:12.212707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:12.212707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:12.212707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:12.212707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:12.212707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:12.212707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:12.212707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:12.212707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 336/680] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:12.213203 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabbc56-a4ba-43a2-b7e7-224574d13946 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:12.236486 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:12.251422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f0fb1-028f-4e1c-a638-5e77d0afebc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:12.252227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f0fb1-028f-4e1c-a638-5e77d0afebc3 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:12.252513 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f0fb1-028f-4e1c-a638-5e77d0afebc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:12.252791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f0fb1-028f-4e1c-a638-5e77d0afebc3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:12.263376 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f0fb1-028f-4e1c-a638-5e77d0afebc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:12.271299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-583f0fb1-028f-4e1c-a638-5e77d0afebc3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:21:12.271299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:12.271299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:12.271299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:12.271299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:12.271299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:12.271299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:12.271299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:12.271299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:12.271299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:12.271299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:12.271299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:12.271299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:12.271299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:12.271299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:12.271299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:12.272230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:12.272230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:12.272230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:12.272230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:12.272230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:12.272230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:12.272230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:12.272230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:12.272230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:12.272230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:12.272230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:12.272230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:12.272230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:12.272230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:12.272230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:12.272230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:12.273173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:12.273173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:12.273173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:12.273173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:12.273173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:12.273173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:12.273173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:12.273173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:12.273173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:12.273173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:12.273173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:21:12.273173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:12.273173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 345/681] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:21:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:12.273889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f0fb1-028f-4e1c-a638-5e77d0afebc3 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:12.302423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:12.329536 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabbc56-a4ba-43a2-b7e7-224574d13946 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:12.367685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eba805-402c-4a78-8069-3b940dd6c628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:12.368701 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eba805-402c-4a78-8069-3b940dd6c628 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:12.368987 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eba805-402c-4a78-8069-3b940dd6c628 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:12.369262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eba805-402c-4a78-8069-3b940dd6c628 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:12.369995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eba805-402c-4a78-8069-3b940dd6c628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:12.370306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eba805-402c-4a78-8069-3b940dd6c628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:12.371710 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eba805-402c-4a78-8069-3b940dd6c628 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:12.384392 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 337/682] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:12.384992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eba805-402c-4a78-8069-3b940dd6c628 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:12.404630 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f0fb1-028f-4e1c-a638-5e77d0afebc3 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:12.445002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea142e47-0ab4-4dc5-b6af-1d31863ea394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:12.445689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea142e47-0ab4-4dc5-b6af-1d31863ea394 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:12.445966 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea142e47-0ab4-4dc5-b6af-1d31863ea394 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:12.446241 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea142e47-0ab4-4dc5-b6af-1d31863ea394 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:12.447141 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea142e47-0ab4-4dc5-b6af-1d31863ea394 None admin] RBAC: Adding query filter params (role.id=ea49c36b4a7f4cfb9048065638d458af, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=01b4142ea6f142df9b1314532c961f75) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:12.447453 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea142e47-0ab4-4dc5-b6af-1d31863ea394 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:12.448924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea142e47-0ab4-4dc5-b6af-1d31863ea394 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:12.456930 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 346/683] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:21:12 2022] GET /identity/v3/role_assignments?user.id=01b4142ea6f142df9b1314532c961f75&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=ea49c36b4a7f4cfb9048065638d458af => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:12.457890 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea142e47-0ab4-4dc5-b6af-1d31863ea394 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:12.490481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:12.523166 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eba805-402c-4a78-8069-3b940dd6c628 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:12.564271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:12.579276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97304b-2bd4-488f-a2f0-c9ff2a5ce283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:12.580184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea142e47-0ab4-4dc5-b6af-1d31863ea394 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:12.580609 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97304b-2bd4-488f-a2f0-c9ff2a5ce283 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:12.581197 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97304b-2bd4-488f-a2f0-c9ff2a5ce283 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:12.581681 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97304b-2bd4-488f-a2f0-c9ff2a5ce283 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:12.584049 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97304b-2bd4-488f-a2f0-c9ff2a5ce283 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:12.584628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97304b-2bd4-488f-a2f0-c9ff2a5ce283 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:12.586319 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97304b-2bd4-488f-a2f0-c9ff2a5ce283 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:12.605138 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 338/684] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:12.606751 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97304b-2bd4-488f-a2f0-c9ff2a5ce283 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:12.630869 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e663b2-105b-43bb-9f48-cd0ea6f24479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:12.631633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e663b2-105b-43bb-9f48-cd0ea6f24479 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:12.632001 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e663b2-105b-43bb-9f48-cd0ea6f24479 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:12.632279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e663b2-105b-43bb-9f48-cd0ea6f24479 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:12.633050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e663b2-105b-43bb-9f48-cd0ea6f24479 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:12.633356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e663b2-105b-43bb-9f48-cd0ea6f24479 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:12.634887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e663b2-105b-43bb-9f48-cd0ea6f24479 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:12.646121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 347/685] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:21:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:12.647749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e663b2-105b-43bb-9f48-cd0ea6f24479 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:12.758346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97304b-2bd4-488f-a2f0-c9ff2a5ce283 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:12.774505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e663b2-105b-43bb-9f48-cd0ea6f24479 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:12.805879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea0d63-4274-47aa-aee2-e8b84d8f1cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:12.806971 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea0d63-4274-47aa-aee2-e8b84d8f1cea None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:12.807252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea0d63-4274-47aa-aee2-e8b84d8f1cea None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:12.807525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea0d63-4274-47aa-aee2-e8b84d8f1cea None admin] PATH_INFO: `/v3/users/swift` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:12.814980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885542b-a5eb-4423-af26-cd5b34a75592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:12.815912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885542b-a5eb-4423-af26-cd5b34a75592 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:12.816475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885542b-a5eb-4423-af26-cd5b34a75592 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:12.817014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea0d63-4274-47aa-aee2-e8b84d8f1cea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:12.817138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885542b-a5eb-4423-af26-cd5b34a75592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:12.818213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885542b-a5eb-4423-af26-cd5b34a75592 None admin] RBAC: Adding query filter params (role.id=ea49c36b4a7f4cfb9048065638d458af, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=f88257850ee34ad895208ab935b6e643) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:12.818452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea0d63-4274-47aa-aee2-e8b84d8f1cea None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:12.818959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885542b-a5eb-4423-af26-cd5b34a75592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:12.820697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885542b-a5eb-4423-af26-cd5b34a75592 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:12.821124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:12.830268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 348/686] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:21:12 2022] GET /identity/v3/role_assignments?user.id=f88257850ee34ad895208ab935b6e643&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=ea49c36b4a7f4cfb9048065638d458af => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:12.831466 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885542b-a5eb-4423-af26-cd5b34a75592 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:12.837333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-84ea0d63-4274-47aa-aee2-e8b84d8f1cea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 20:21:12.837333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:12.837333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:12.837333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:12.837333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:12.837333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:12.837333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:12.837333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:12.837333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:12.837333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:12.837333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:12.837333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:12.837333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:12.837333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:12.837333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:12.837333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:12.838367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:12.838367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:12.838367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:12.838367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:12.838367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:12.838367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:12.838367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:12.838367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:12.838367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:12.838367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:12.838367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:12.838367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:12.838367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:12.838367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:12.838367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:12.838367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:12.838367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:12.838367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:12.839393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:12.839393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:12.839393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:12.839393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:12.839393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:12.839393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:12.839393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:12.839393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:12.839393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 20:21:12.839393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:12.844684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 339/687] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:21:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:12.850593 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea0d63-4274-47aa-aee2-e8b84d8f1cea None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:12.851725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:12.965722 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885542b-a5eb-4423-af26-cd5b34a75592 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:12.973150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea0d63-4274-47aa-aee2-e8b84d8f1cea None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:13.031310 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b858e04-971d-4fd9-b2ff-4a2f40dbe9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:13.031944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b858e04-971d-4fd9-b2ff-4a2f40dbe9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:13.032186 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b858e04-971d-4fd9-b2ff-4a2f40dbe9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:13.034962 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b858e04-971d-4fd9-b2ff-4a2f40dbe9e9 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:13.034962 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b858e04-971d-4fd9-b2ff-4a2f40dbe9e9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:13.034962 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b858e04-971d-4fd9-b2ff-4a2f40dbe9e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:13.035244 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b858e04-971d-4fd9-b2ff-4a2f40dbe9e9 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:13.040426 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe9e79-52cc-426a-9d40-6f7bcf30a9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:13.041246 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe9e79-52cc-426a-9d40-6f7bcf30a9de None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:13.041534 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe9e79-52cc-426a-9d40-6f7bcf30a9de None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:13.041812 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe9e79-52cc-426a-9d40-6f7bcf30a9de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:13.059476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe9e79-52cc-426a-9d40-6f7bcf30a9de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:13.061582 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe9e79-52cc-426a-9d40-6f7bcf30a9de None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:13.075464 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 340/688] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jul 14 20:21:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:13.082126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b858e04-971d-4fd9-b2ff-4a2f40dbe9e9 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:13.093319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-7dfe9e79-52cc-426a-9d40-6f7bcf30a9de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:13.093319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:13.093319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:13.093319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:13.093319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:13.093319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:13.093319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:13.093319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:13.093319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:13.093319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:13.093319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:13.093319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:13.093319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:13.093319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:13.093319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:13.094252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:13.094252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:13.094252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:13.094252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:13.094252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:13.094252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:13.094252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:13.094252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:13.094252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:13.094252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:13.094252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:13.094252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:13.094252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:13.094252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:13.094252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:13.094252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:13.094252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:13.095165 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:13.095165 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:13.095165 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:13.095165 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:13.095878 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 349/689] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:13.102337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe9e79-52cc-426a-9d40-6f7bcf30a9de None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:13.159295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:13.182090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:13.207891 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b858e04-971d-4fd9-b2ff-4a2f40dbe9e9 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:13.224141 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe9e79-52cc-426a-9d40-6f7bcf30a9de None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:13.246468 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c284f-4e91-4a7d-a0d6-df1eef13eb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:13.247962 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c284f-4e91-4a7d-a0d6-df1eef13eb68 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:13.248400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c284f-4e91-4a7d-a0d6-df1eef13eb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:13.248851 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c284f-4e91-4a7d-a0d6-df1eef13eb68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:13.263481 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1ba58-fdeb-4a4b-ae1e-699b2b445312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:13.264157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1ba58-fdeb-4a4b-ae1e-699b2b445312 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:13.264444 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1ba58-fdeb-4a4b-ae1e-699b2b445312 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:13.264717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1ba58-fdeb-4a4b-ae1e-699b2b445312 None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:13.265457 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1ba58-fdeb-4a4b-ae1e-699b2b445312 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:13.265764 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1ba58-fdeb-4a4b-ae1e-699b2b445312 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:13.267470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1ba58-fdeb-4a4b-ae1e-699b2b445312 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:13.283499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 350/690] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:13.289080 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1ba58-fdeb-4a4b-ae1e-699b2b445312 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:13.329080 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c284f-4e91-4a7d-a0d6-df1eef13eb68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:13.331097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c284f-4e91-4a7d-a0d6-df1eef13eb68 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:13.394958 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:13.402064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1ba58-fdeb-4a4b-ae1e-699b2b445312 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:13.431288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:13.441315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ba4e2-e4a2-41e2-bc53-81b0039c7d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:13.442329 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ba4e2-e4a2-41e2-bc53-81b0039c7d69 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:13.442517 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ba4e2-e4a2-41e2-bc53-81b0039c7d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:13.442772 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ba4e2-e4a2-41e2-bc53-81b0039c7d69 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:13.446857 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 341/691] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:21:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 20:21:13.450710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ba4e2-e4a2-41e2-bc53-81b0039c7d69 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:13.452256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c284f-4e91-4a7d-a0d6-df1eef13eb68 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:13.452373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ba4e2-e4a2-41e2-bc53-81b0039c7d69 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:13.469433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-dc3ba4e2-e4a2-41e2-bc53-81b0039c7d69 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 20:21:13.469433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:13.469433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:13.469433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:13.469433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:13.469433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:13.469433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:13.469433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:13.469433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:13.469433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:13.469433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:13.469433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:13.469433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:13.469433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:13.469433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:13.469433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:13.470458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:13.470458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:13.470458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:13.470458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:13.470458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:13.470458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:13.470458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:13.470458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:13.470458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:13.470458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:13.470458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:13.470458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:13.470458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:13.470458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:13.470458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:13.470458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:13.470458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:13.470458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:13.471425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:13.471425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:13.471425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:13.471425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:13.471425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:13.471425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:13.471425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:13.471425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:13.471425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 20:21:13.471425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:13.475065 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 351/692] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 14 20:21:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:21:13.476502 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ba4e2-e4a2-41e2-bc53-81b0039c7d69 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:13.581014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c284f-4e91-4a7d-a0d6-df1eef13eb68 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:13.600885 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ba4e2-e4a2-41e2-bc53-81b0039c7d69 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:13.642700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fadf73-3ea9-4655-9ea7-7e67ab9274f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:13.643530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fadf73-3ea9-4655-9ea7-7e67ab9274f2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:13.643806 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fadf73-3ea9-4655-9ea7-7e67ab9274f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:13.646205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fadf73-3ea9-4655-9ea7-7e67ab9274f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:13.658679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227df62a-202c-40ed-bda3-2f023b6ad621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:13.659402 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227df62a-202c-40ed-bda3-2f023b6ad621 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:13.659685 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227df62a-202c-40ed-bda3-2f023b6ad621 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:13.662968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227df62a-202c-40ed-bda3-2f023b6ad621 None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:13.662968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227df62a-202c-40ed-bda3-2f023b6ad621 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:13.662968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227df62a-202c-40ed-bda3-2f023b6ad621 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:13.662968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227df62a-202c-40ed-bda3-2f023b6ad621 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:13.680128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fadf73-3ea9-4655-9ea7-7e67ab9274f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:13.682049 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fadf73-3ea9-4655-9ea7-7e67ab9274f2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:13.682271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 352/693] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 14 20:21:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:13.689284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227df62a-202c-40ed-bda3-2f023b6ad621 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:13.714328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-20fadf73-3ea9-4655-9ea7-7e67ab9274f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:13.714328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:13.714328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:13.714328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:13.714328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:13.714328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:13.714328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:13.714328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:13.714328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:13.714328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:13.714328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:13.714328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:13.714328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:13.714328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:13.714328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:13.715311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:13.715311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:13.715311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:13.715311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:13.715311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:13.715311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:13.715311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:13.715311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:13.715311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:13.715311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:13.715311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:13.715311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:13.715311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:13.715311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:13.715311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:13.715311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:13.715311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:13.716706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:13.716706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:13.716706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:13.716706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:13.723069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 342/694] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:13.724775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b05239-220e-45a9-b848-7d8f7b62bf55 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:13.725273 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b05239-220e-45a9-b848-7d8f7b62bf55 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:13.725765 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b05239-220e-45a9-b848-7d8f7b62bf55 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:13.727584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 343/695] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:13.729225 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b05239-220e-45a9-b848-7d8f7b62bf55 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:13.746041 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:13.754072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:13.823407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227df62a-202c-40ed-bda3-2f023b6ad621 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:13.862604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b05239-220e-45a9-b848-7d8f7b62bf55 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:13.869950 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544f061-347b-43eb-bdf9-7257f9a28b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:13.870614 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544f061-347b-43eb-bdf9-7257f9a28b7c None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:13.870900 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544f061-347b-43eb-bdf9-7257f9a28b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:13.871611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544f061-347b-43eb-bdf9-7257f9a28b7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:13.900050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20918268-2067-4dfa-accf-a847e8b8f850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:13.901213 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20918268-2067-4dfa-accf-a847e8b8f850 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:13.901815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20918268-2067-4dfa-accf-a847e8b8f850 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:13.902287 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20918268-2067-4dfa-accf-a847e8b8f850 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:13.903284 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20918268-2067-4dfa-accf-a847e8b8f850 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:13.904455 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20918268-2067-4dfa-accf-a847e8b8f850 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:13.906170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20918268-2067-4dfa-accf-a847e8b8f850 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:13.924580 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 344/696] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:13.926512 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b8bfe-ab85-45f2-a403-a0f9cfadb244 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:13.927030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b8bfe-ab85-45f2-a403-a0f9cfadb244 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:13.927541 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b8bfe-ab85-45f2-a403-a0f9cfadb244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:13.945833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544f061-347b-43eb-bdf9-7257f9a28b7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:13.946863 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-534b8bfe-ab85-45f2-a403-a0f9cfadb244 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:13.947345 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544f061-347b-43eb-bdf9-7257f9a28b7c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:13.948584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-534b8bfe-ab85-45f2-a403-a0f9cfadb244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:13.999738 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:14.023541 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:14.029619 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 345/697] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:21:14.030880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b8bfe-ab85-45f2-a403-a0f9cfadb244 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:14.089642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 353/698] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:21:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 20:21:14.090967 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544f061-347b-43eb-bdf9-7257f9a28b7c None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:14.138228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b8bfe-ab85-45f2-a403-a0f9cfadb244 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:14.177546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07eabe-546b-45f8-9e17-3feded788823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:14.178702 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07eabe-546b-45f8-9e17-3feded788823 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:14.179230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07eabe-546b-45f8-9e17-3feded788823 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:14.179705 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07eabe-546b-45f8-9e17-3feded788823 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:14.188091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07eabe-546b-45f8-9e17-3feded788823 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:14.190048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07eabe-546b-45f8-9e17-3feded788823 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:14.208801 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544f061-347b-43eb-bdf9-7257f9a28b7c None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:14.209156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-1e07eabe-546b-45f8-9e17-3feded788823 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:14.209156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:14.209156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:14.209156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:14.209156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:14.209156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:14.209156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:14.209156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:14.209156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:14.209156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:14.209156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:14.209156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:14.209156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:14.209156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:14.209156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:14.210051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:14.210051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:14.210051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:14.210051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:14.210051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:14.210051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:14.210051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:14.210051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:14.210051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:14.210051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:14.210051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:14.210051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:14.210051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:14.210051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:14.210051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:14.210051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:14.210051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:14.210995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:14.210995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:14.210995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:14.210995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:14.210995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:14.210995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:14.210995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:14.210995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:14.212399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 346/699] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:14.218176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07eabe-546b-45f8-9e17-3feded788823 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:14.256253 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39387c1f-3310-4724-a37d-246a471a7f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:14.257226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39387c1f-3310-4724-a37d-246a471a7f6f None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:14.257523 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39387c1f-3310-4724-a37d-246a471a7f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:14.257807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39387c1f-3310-4724-a37d-246a471a7f6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:14.271780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39387c1f-3310-4724-a37d-246a471a7f6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:14.273330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39387c1f-3310-4724-a37d-246a471a7f6f None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:14.277248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:14.300247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-39387c1f-3310-4724-a37d-246a471a7f6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:14.300247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:14.300247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:14.300247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:14.300247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:14.300247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:14.300247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:14.300247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:14.300247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:14.300247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:14.300247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:14.300247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:14.300247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:14.300247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:14.300247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:14.301385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:14.301385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:14.301385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:14.301385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:14.301385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:14.301385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:14.301385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:14.301385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:14.301385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:14.301385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:14.301385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:14.301385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:14.301385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:14.301385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:14.301385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:14.301385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:14.301385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:14.302296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:14.302296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:14.302296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:14.302296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:14.302296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 354/700] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:21:14.303051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39387c1f-3310-4724-a37d-246a471a7f6f None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:14.305891 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:14.368439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07eabe-546b-45f8-9e17-3feded788823 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:14.408362 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1c4c2-9208-4e4d-bfe4-d82d50374237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:14.409893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1c4c2-9208-4e4d-bfe4-d82d50374237 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:14.410191 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1c4c2-9208-4e4d-bfe4-d82d50374237 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:14.410478 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1c4c2-9208-4e4d-bfe4-d82d50374237 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:14.425700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d1c4c2-9208-4e4d-bfe4-d82d50374237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:14.427520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d1c4c2-9208-4e4d-bfe4-d82d50374237 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:14.450547 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39387c1f-3310-4724-a37d-246a471a7f6f None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:14.453103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-67d1c4c2-9208-4e4d-bfe4-d82d50374237 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:14.453103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:14.453103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:14.453103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:14.453103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:14.453103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:14.453103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:14.453103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:14.453103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:14.453103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:14.453103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:14.453103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:14.453103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:14.453103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:14.453103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:14.453977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:14.453977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:14.453977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:14.453977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:14.453977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:14.453977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:14.453977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:14.453977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:14.453977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:14.453977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:14.453977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:14.453977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:14.453977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:14.453977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:14.453977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:14.453977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:14.453977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:14.454854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:14.454854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:14.454854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:14.454854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:14.454854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 347/701] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:14.455546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1c4c2-9208-4e4d-bfe4-d82d50374237 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:14.502051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f69f64-4e80-4cd6-9cf1-caaa83224007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:14.503300 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f69f64-4e80-4cd6-9cf1-caaa83224007 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:14.503885 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f69f64-4e80-4cd6-9cf1-caaa83224007 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:14.504474 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f69f64-4e80-4cd6-9cf1-caaa83224007 None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:14.505546 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f69f64-4e80-4cd6-9cf1-caaa83224007 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:14.506071 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f69f64-4e80-4cd6-9cf1-caaa83224007 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:14.507751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f69f64-4e80-4cd6-9cf1-caaa83224007 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:14.528094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 355/702] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:14.528094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f69f64-4e80-4cd6-9cf1-caaa83224007 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:14.588643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:14.590667 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1c4c2-9208-4e4d-bfe4-d82d50374237 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:14.608708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:14.643498 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fda75-06ac-49c0-966f-d0664fe27274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:14.644572 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fda75-06ac-49c0-966f-d0664fe27274 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:14.644863 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fda75-06ac-49c0-966f-d0664fe27274 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:14.645141 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fda75-06ac-49c0-966f-d0664fe27274 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:14.646016 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fda75-06ac-49c0-966f-d0664fe27274 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:14.646336 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fda75-06ac-49c0-966f-d0664fe27274 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:14.648324 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fda75-06ac-49c0-966f-d0664fe27274 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:14.649972 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f69f64-4e80-4cd6-9cf1-caaa83224007 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:14.664288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 348/703] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:14.665319 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fda75-06ac-49c0-966f-d0664fe27274 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:14.691106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ddaf26-01a2-4421-a6fc-f95c9ce2661d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:14.691973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ddaf26-01a2-4421-a6fc-f95c9ce2661d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:14.692382 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ddaf26-01a2-4421-a6fc-f95c9ce2661d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:14.692808 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ddaf26-01a2-4421-a6fc-f95c9ce2661d None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:14.693656 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ddaf26-01a2-4421-a6fc-f95c9ce2661d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:14.694160 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ddaf26-01a2-4421-a6fc-f95c9ce2661d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:14.695946 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ddaf26-01a2-4421-a6fc-f95c9ce2661d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:14.712246 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 356/704] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:14.713293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ddaf26-01a2-4421-a6fc-f95c9ce2661d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:14.784502 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fda75-06ac-49c0-966f-d0664fe27274 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:14.824103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65a6eb-c29d-4144-a305-3e944ff50e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:14.825110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65a6eb-c29d-4144-a305-3e944ff50e62 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:14.826148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65a6eb-c29d-4144-a305-3e944ff50e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:14.826613 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65a6eb-c29d-4144-a305-3e944ff50e62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:14.830998 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65a6eb-c29d-4144-a305-3e944ff50e62 None admin] RBAC: Adding query filter params (role.id=ea49c36b4a7f4cfb9048065638d458af, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=acc23233929144a5b7b3a1733da19828) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:14.830998 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65a6eb-c29d-4144-a305-3e944ff50e62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:14.830998 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65a6eb-c29d-4144-a305-3e944ff50e62 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:14.842968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ddaf26-01a2-4421-a6fc-f95c9ce2661d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:14.847202 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 349/705] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:21:14 2022] GET /identity/v3/role_assignments?user.id=acc23233929144a5b7b3a1733da19828&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=ea49c36b4a7f4cfb9048065638d458af => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:14.849532 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65a6eb-c29d-4144-a305-3e944ff50e62 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:14.879318 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:14.923443 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacfc634-f455-4d48-af55-16ebd08ddf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:14.924068 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacfc634-f455-4d48-af55-16ebd08ddf4f None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:14.926794 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacfc634-f455-4d48-af55-16ebd08ddf4f None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:14.927197 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacfc634-f455-4d48-af55-16ebd08ddf4f None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:14.927980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacfc634-f455-4d48-af55-16ebd08ddf4f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:14.929483 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacfc634-f455-4d48-af55-16ebd08ddf4f None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:14.942583 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:14.960965 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 357/706] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 14 20:21:14 2022] POST /identity/v3/users => generated 254 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:21:14.962660 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e406264-96d3-4b65-931f-dc8f6a6d4b35 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:14.963175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e406264-96d3-4b65-931f-dc8f6a6d4b35 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:14.963604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e406264-96d3-4b65-931f-dc8f6a6d4b35 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:14.965652 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 358/707] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:14.984937 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a9f23-d317-4c2f-9439-d64fd325aae7 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:14.985512 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a9f23-d317-4c2f-9439-d64fd325aae7 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:14.985941 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a9f23-d317-4c2f-9439-d64fd325aae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:15.017624 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-fe2a9f23-d317-4c2f-9439-d64fd325aae7 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:15.019110 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-fe2a9f23-d317-4c2f-9439-d64fd325aae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:15.035696 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65a6eb-c29d-4144-a305-3e944ff50e62 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:15.084567 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294a787-97d0-467b-89d3-24cde115ffb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:15.086698 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294a787-97d0-467b-89d3-24cde115ffb1 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:15.087178 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294a787-97d0-467b-89d3-24cde115ffb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:15.087509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294a787-97d0-467b-89d3-24cde115ffb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:15.099897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f294a787-97d0-467b-89d3-24cde115ffb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:15.101996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f294a787-97d0-467b-89d3-24cde115ffb1 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:15.120849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-f294a787-97d0-467b-89d3-24cde115ffb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:15.120849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:15.120849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:15.120849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:15.120849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:15.120849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:15.120849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:15.120849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:15.120849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:15.120849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:15.120849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:15.120849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:15.120849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:15.120849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:15.120849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:15.121815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:15.121815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:15.121815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:15.121815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:15.121815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:15.121815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:15.121815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:15.121815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:15.121815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:15.121815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:15.121815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:15.121815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:15.121815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:15.121815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:15.121815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:15.121815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:15.121815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:15.123275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:15.123275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:15.123275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:15.123275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:15.123275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:15.123275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:15.123275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:15.123275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:15.123275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 350/708] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:15.129146 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294a787-97d0-467b-89d3-24cde115ffb1 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:15.135041 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 359/709] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:21:15.162146 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a9f23-d317-4c2f-9439-d64fd325aae7 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:15.253424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:15.265763 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294a787-97d0-467b-89d3-24cde115ffb1 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:15.291149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a9f23-d317-4c2f-9439-d64fd325aae7 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:15.293303 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:15.312798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cb51b-bd82-4018-8a95-3d91197ea8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:15.313823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cb51b-bd82-4018-8a95-3d91197ea8ee None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:15.314151 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cb51b-bd82-4018-8a95-3d91197ea8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:15.314599 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cb51b-bd82-4018-8a95-3d91197ea8ee None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:15.315753 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cb51b-bd82-4018-8a95-3d91197ea8ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:15.316070 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cb51b-bd82-4018-8a95-3d91197ea8ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:15.317545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cb51b-bd82-4018-8a95-3d91197ea8ee None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:15.334727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001041b-c5de-4872-a6db-399b641be680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:15.335807 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 351/710] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:15.336244 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001041b-c5de-4872-a6db-399b641be680 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:15.336589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001041b-c5de-4872-a6db-399b641be680 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:15.336932 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001041b-c5de-4872-a6db-399b641be680 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:15.337693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001041b-c5de-4872-a6db-399b641be680 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:15.339240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001041b-c5de-4872-a6db-399b641be680 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:15.341706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cb51b-bd82-4018-8a95-3d91197ea8ee None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:15.354873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-4001041b-c5de-4872-a6db-399b641be680 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 20:21:15.354873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:15.354873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:15.354873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:15.354873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:15.354873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:15.354873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:15.354873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:15.354873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 20:21:15.354873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:15.354873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:15.354873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:15.354873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:15.354873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:15.354873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:15.354873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:15.355964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:15.355964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:15.355964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:15.355964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:15.355964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:15.355964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:15.355964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:15.355964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:15.355964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:15.355964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:15.355964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:15.355964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:15.355964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:15.355964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:15.355964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:15.355964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:15.356854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:15.356854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:15.356854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:15.356854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:15.356854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:15.356854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:15.356854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:15.356854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:15.356854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:15.356854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:15.356854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:15.356854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:15.356854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:15.356854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 20:21:15.356854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:15.356854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 360/711] 10.222.0.52 () {56 vars in 1129 bytes} [Thu Jul 14 20:21:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:21:15.362724 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001041b-c5de-4872-a6db-399b641be680 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:15.481393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cb51b-bd82-4018-8a95-3d91197ea8ee None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:15.493110 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001041b-c5de-4872-a6db-399b641be680 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:15.539364 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e928c-1fff-4fa5-bf03-2646f83a5060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:15.540110 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ea261-173a-4481-b556-36b79f03d87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:15.540465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e928c-1fff-4fa5-bf03-2646f83a5060 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:15.540933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e928c-1fff-4fa5-bf03-2646f83a5060 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:15.541061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ea261-173a-4481-b556-36b79f03d87e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:15.541430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ea261-173a-4481-b556-36b79f03d87e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:15.541806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ea261-173a-4481-b556-36b79f03d87e None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:15.541916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e928c-1fff-4fa5-bf03-2646f83a5060 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:15.542713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ea261-173a-4481-b556-36b79f03d87e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:15.543123 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e928c-1fff-4fa5-bf03-2646f83a5060 None admin] RBAC: Adding query filter params (role.id=5ca112885467422684e134cb0f94865a, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=aa3bb9f1e24a4e9ebd460e97b3736bc0) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:15.543252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ea261-173a-4481-b556-36b79f03d87e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:15.543724 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e928c-1fff-4fa5-bf03-2646f83a5060 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:15.545331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ea261-173a-4481-b556-36b79f03d87e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:15.546036 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e928c-1fff-4fa5-bf03-2646f83a5060 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:15.550838 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 361/712] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jul 14 20:21:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:15.552189 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8942a6-c0e6-4acc-8c02-fd4135f112dc None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:15.552578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8942a6-c0e6-4acc-8c02-fd4135f112dc None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:15.552961 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8942a6-c0e6-4acc-8c02-fd4135f112dc None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:15.553783 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 352/713] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:21:15 2022] GET /identity/v3/role_assignments?user.id=aa3bb9f1e24a4e9ebd460e97b3736bc0&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=5ca112885467422684e134cb0f94865a => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:15.554575 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 362/714] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:15.563097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e928c-1fff-4fa5-bf03-2646f83a5060 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:15.563334 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daf5ba-ea3d-429c-9a84-8a173eb99bd5 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:15.563334 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daf5ba-ea3d-429c-9a84-8a173eb99bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:15.563334 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daf5ba-ea3d-429c-9a84-8a173eb99bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:15.580089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-d7daf5ba-ea3d-429c-9a84-8a173eb99bd5 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:15.581479 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d7daf5ba-ea3d-429c-9a84-8a173eb99bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:15.608743 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:15.609737 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:15.664707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 363/715] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 20:21:15.688376 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7daf5ba-ea3d-429c-9a84-8a173eb99bd5 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:15.807487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e928c-1fff-4fa5-bf03-2646f83a5060 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:15.835726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7daf5ba-ea3d-429c-9a84-8a173eb99bd5 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:15.872957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516c551-c242-49a1-ab3b-be3575275b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:15.873638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516c551-c242-49a1-ab3b-be3575275b51 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:15.873923 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516c551-c242-49a1-ab3b-be3575275b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:15.874202 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516c551-c242-49a1-ab3b-be3575275b51 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:15.874923 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516c551-c242-49a1-ab3b-be3575275b51 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:15.875258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516c551-c242-49a1-ab3b-be3575275b51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:15.876633 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516c551-c242-49a1-ab3b-be3575275b51 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:15.894385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 353/716] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jul 14 20:21:15 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:15.914299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8329a21-9a07-498e-9fe8-260756361996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:15.914299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8329a21-9a07-498e-9fe8-260756361996 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:15.914299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8329a21-9a07-498e-9fe8-260756361996 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:15.914299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8329a21-9a07-498e-9fe8-260756361996 None admin] PATH_INFO: `/v3/services/network` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:15.914721 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8329a21-9a07-498e-9fe8-260756361996 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:15.915024 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8329a21-9a07-498e-9fe8-260756361996 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:15.938094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-d8329a21-9a07-498e-9fe8-260756361996 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 20:21:15.938094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:15.938094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:15.938094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:15.938094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:15.938094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:15.938094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:15.938094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:15.938094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 20:21:15.938094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:15.938094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:15.938094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:15.938094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:15.938094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:15.938094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:15.938094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:15.938094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:15.939232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:15.939232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:15.939232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:15.939232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:15.939232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:15.939232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:15.939232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:15.939232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:15.939232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:15.939232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:15.939232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:15.939232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:15.939232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:15.939232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:15.939232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:15.939232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:15.940096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:15.940096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:15.940096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:15.940096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:15.940096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:15.940096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:15.940096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:15.940096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:15.940096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:15.940096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:15.940096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:15.940096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:15.940096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 20:21:15.940096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:15.940096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 364/717] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:15 2022] GET /identity/v3/services/network => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:21:15.948141 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516c551-c242-49a1-ab3b-be3575275b51 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:16.088054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:16.107662 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516c551-c242-49a1-ab3b-be3575275b51 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:16.149197 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18331b0-67d2-400c-a072-4c9261c37286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:16.149868 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18331b0-67d2-400c-a072-4c9261c37286 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:16.150148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18331b0-67d2-400c-a072-4c9261c37286 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:16.150424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18331b0-67d2-400c-a072-4c9261c37286 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:16.151171 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18331b0-67d2-400c-a072-4c9261c37286 None admin] RBAC: Adding query filter params (name=network) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:16.151477 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18331b0-67d2-400c-a072-4c9261c37286 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:16.152858 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18331b0-67d2-400c-a072-4c9261c37286 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:16.163102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 354/718] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 14 20:21:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:16.168602 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8329a21-9a07-498e-9fe8-260756361996 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:16.229086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:16.321817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8329a21-9a07-498e-9fe8-260756361996 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:16.380272 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7384f-a632-4a81-a989-a073d9926afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:16.380900 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7384f-a632-4a81-a989-a073d9926afe None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:16.381187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7384f-a632-4a81-a989-a073d9926afe None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:16.381465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7384f-a632-4a81-a989-a073d9926afe None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:16.382189 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7384f-a632-4a81-a989-a073d9926afe None admin] RBAC: Adding query filter params (type=network) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:16.382499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7384f-a632-4a81-a989-a073d9926afe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:16.383899 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7384f-a632-4a81-a989-a073d9926afe None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:16.398152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 365/719] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 14 20:21:16 2022] GET /identity/v3/services?type=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:17.708976 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879057b6-b276-40af-9a1e-17ca2956e3ab None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:17.709529 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879057b6-b276-40af-9a1e-17ca2956e3ab None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:17.709529 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879057b6-b276-40af-9a1e-17ca2956e3ab None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:17.711224 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 355/720] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:17.716837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa88c04-76ee-41bc-86ad-ec8d07be4ea5 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:17.717273 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa88c04-76ee-41bc-86ad-ec8d07be4ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:17.717469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa88c04-76ee-41bc-86ad-ec8d07be4ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:17.738630 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-5fa88c04-76ee-41bc-86ad-ec8d07be4ea5 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:17.739849 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-5fa88c04-76ee-41bc-86ad-ec8d07be4ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:17.758915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87e0ec-7069-4f2d-b816-9afc2cac7079 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:17.759224 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87e0ec-7069-4f2d-b816-9afc2cac7079 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:17.759503 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87e0ec-7069-4f2d-b816-9afc2cac7079 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:17.761098 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 356/721] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:17.766586 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619bd0a-256c-4e7e-92d7-765b52a918c7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:17.766876 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619bd0a-256c-4e7e-92d7-765b52a918c7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:17.767209 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619bd0a-256c-4e7e-92d7-765b52a918c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:17.802549 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1619bd0a-256c-4e7e-92d7-765b52a918c7 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:17.803786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1619bd0a-256c-4e7e-92d7-765b52a918c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:17.856071 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 366/722] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:21:17.896672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa88c04-76ee-41bc-86ad-ec8d07be4ea5 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:17.916641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:17.926264 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 357/723] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:21:17.944256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e364c-0e42-4a82-8c69-425f28fd439e None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:17.945245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e364c-0e42-4a82-8c69-425f28fd439e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:17.945732 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e364c-0e42-4a82-8c69-425f28fd439e None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:17.947913 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 358/724] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:21:17.954228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8660e-c8ec-46de-b479-3e5e3faf5661 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:17.954792 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8660e-c8ec-46de-b479-3e5e3faf5661 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:17.955334 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8660e-c8ec-46de-b479-3e5e3faf5661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:17.974542 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8ec8660e-c8ec-46de-b479-3e5e3faf5661 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:17.975785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8ec8660e-c8ec-46de-b479-3e5e3faf5661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:18.035487 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa88c04-76ee-41bc-86ad-ec8d07be4ea5 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:18.051410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 359/725] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:21:18.051410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8660e-c8ec-46de-b479-3e5e3faf5661 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:18.063420 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:18.099393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417df6c-cd90-4837-8680-5321405ba12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:18.100213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417df6c-cd90-4837-8680-5321405ba12a None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:18.100500 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417df6c-cd90-4837-8680-5321405ba12a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:18.100769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417df6c-cd90-4837-8680-5321405ba12a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:18.114644 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417df6c-cd90-4837-8680-5321405ba12a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:18.123855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-c417df6c-cd90-4837-8680-5321405ba12a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:21:18.123855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:18.123855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:18.123855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:18.123855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:18.123855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:18.123855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:18.123855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:18.123855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:18.123855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:18.123855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:18.123855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:18.123855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:18.123855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:18.123855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:18.123855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:18.126035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:18.126035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:18.126035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:18.126035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:18.126035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:18.126035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:18.126035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:18.126035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:18.126035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:18.126035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:18.126035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:18.126035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:18.126035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:18.126035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:18.126035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:18.126035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:18.128002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:18.128002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:18.128002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:18.128002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:18.128002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:18.128002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:18.128002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:18.128002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:18.128002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:18.128002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:18.128002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:21:18.128002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:18.128002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 367/726] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:21:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:21:18.128002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417df6c-cd90-4837-8680-5321405ba12a None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:18.216862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8660e-c8ec-46de-b479-3e5e3faf5661 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:18.230529 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:18.257028 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd5456-08c1-47f4-af71-2519c71a9b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:18.258287 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd5456-08c1-47f4-af71-2519c71a9b82 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:18.258793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd5456-08c1-47f4-af71-2519c71a9b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:18.259296 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd5456-08c1-47f4-af71-2519c71a9b82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:18.260475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417df6c-cd90-4837-8680-5321405ba12a None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:18.272915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd5456-08c1-47f4-af71-2519c71a9b82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:18.282759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-16dd5456-08c1-47f4-af71-2519c71a9b82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:18.282759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:18.282759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:18.282759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:18.282759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:18.282759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:18.282759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:18.282759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:18.282759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:18.282759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:18.282759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:18.282759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:18.282759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:18.282759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:18.282759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:18.282759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:18.283823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:18.283823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:18.283823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:18.283823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:18.283823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:18.283823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:18.283823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:18.283823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:18.283823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:18.283823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:18.283823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:18.283823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:18.283823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:18.283823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:18.283823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:18.283823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:18.284677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:18.284677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:18.284677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:18.284677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:18.284677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:18.284677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:18.284677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:18.284677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:18.284677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:18.284677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:18.284677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:18.284677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:18.292168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 360/727] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:21:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:18.293486 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd5456-08c1-47f4-af71-2519c71a9b82 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:18.300822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09b657-cf7f-4860-bf94-6edbbaf706b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:18.301652 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09b657-cf7f-4860-bf94-6edbbaf706b3 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:18.301938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09b657-cf7f-4860-bf94-6edbbaf706b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:18.302217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09b657-cf7f-4860-bf94-6edbbaf706b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:18.316862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09b657-cf7f-4860-bf94-6edbbaf706b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:18.326522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-5c09b657-cf7f-4860-bf94-6edbbaf706b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:21:18.326522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:18.326522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:18.326522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:18.326522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:18.326522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:18.326522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:18.326522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:18.326522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:18.326522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:18.326522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:18.326522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:18.326522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:18.326522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:18.326522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:18.326522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:18.327852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:18.327852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:18.327852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:18.327852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:18.327852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:18.327852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:18.327852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:18.327852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:18.327852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:18.327852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:18.327852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:18.327852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:18.327852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:18.327852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:18.327852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:18.327852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:18.329663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:18.329663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:18.329663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:18.329663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:18.329663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:18.329663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:18.329663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:18.329663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:18.329663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:18.329663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:18.329663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:21:18.329663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:18.332996 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 368/728] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:21:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:18.333911 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09b657-cf7f-4860-bf94-6edbbaf706b3 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:18.379837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:18.421671 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd5456-08c1-47f4-af71-2519c71a9b82 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:18.455356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09b657-cf7f-4860-bf94-6edbbaf706b3 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:18.460887 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405841f-d48e-4c8f-bbc7-273d878a8ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:18.461571 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405841f-d48e-4c8f-bbc7-273d878a8ffa None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:18.461857 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405841f-d48e-4c8f-bbc7-273d878a8ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:18.462191 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405841f-d48e-4c8f-bbc7-273d878a8ffa None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:18.462968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2405841f-d48e-4c8f-bbc7-273d878a8ffa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:18.463285 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2405841f-d48e-4c8f-bbc7-273d878a8ffa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:18.464688 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2405841f-d48e-4c8f-bbc7-273d878a8ffa None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:18.474677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 361/729] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:21:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:18.475455 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405841f-d48e-4c8f-bbc7-273d878a8ffa None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:18.498781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2c4d2-0e85-4566-b534-6a51f44745b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:18.499601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2c4d2-0e85-4566-b534-6a51f44745b6 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:18.499889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2c4d2-0e85-4566-b534-6a51f44745b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:18.500164 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2c4d2-0e85-4566-b534-6a51f44745b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:18.500922 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2c4d2-0e85-4566-b534-6a51f44745b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:18.501233 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2c4d2-0e85-4566-b534-6a51f44745b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:18.502716 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2c4d2-0e85-4566-b534-6a51f44745b6 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:18.508794 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:18.514173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 369/730] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:21:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:18.515480 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374e3a5-b1bc-4d5f-98f5-cf7ef7901df1 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:18.516081 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374e3a5-b1bc-4d5f-98f5-cf7ef7901df1 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:18.516689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374e3a5-b1bc-4d5f-98f5-cf7ef7901df1 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:18.522192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 370/731] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:18.522192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374e3a5-b1bc-4d5f-98f5-cf7ef7901df1 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:18.588767 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405841f-d48e-4c8f-bbc7-273d878a8ffa None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:18.619840 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:18.626943 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2748ef-5c09-4afb-a8eb-7495200c977b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:18.627639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2748ef-5c09-4afb-a8eb-7495200c977b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:18.627925 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2748ef-5c09-4afb-a8eb-7495200c977b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:18.628204 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2748ef-5c09-4afb-a8eb-7495200c977b None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:18.628965 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2748ef-5c09-4afb-a8eb-7495200c977b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:18.629271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2748ef-5c09-4afb-a8eb-7495200c977b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:18.630804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2748ef-5c09-4afb-a8eb-7495200c977b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:18.638329 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374e3a5-b1bc-4d5f-98f5-cf7ef7901df1 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:18.648057 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 362/732] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:21:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:18.648057 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2748ef-5c09-4afb-a8eb-7495200c977b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:18.676478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef563196-791a-4804-8bdb-ce28e7def135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:18.677375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef563196-791a-4804-8bdb-ce28e7def135 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:18.677726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef563196-791a-4804-8bdb-ce28e7def135 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:18.678068 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef563196-791a-4804-8bdb-ce28e7def135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:18.690902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef563196-791a-4804-8bdb-ce28e7def135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:18.692434 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef563196-791a-4804-8bdb-ce28e7def135 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:18.712006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-ef563196-791a-4804-8bdb-ce28e7def135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:18.712006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:18.712006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:18.712006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:18.712006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:18.712006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:18.712006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:18.712006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:18.712006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:18.712006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:18.712006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:18.712006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:18.712006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:18.712006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:18.712006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:18.713113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:18.713113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:18.713113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:18.713113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:18.713113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:18.713113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:18.713113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:18.713113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:18.713113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:18.713113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:18.713113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:18.713113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:18.713113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:18.713113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:18.713113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:18.713113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:18.713113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:18.714031 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:18.714031 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:18.714031 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:18.714031 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:18.714031 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 371/733] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:18.714404 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40f460-0370-46b4-b7b8-4b0b8fbcdba4 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:18.714684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40f460-0370-46b4-b7b8-4b0b8fbcdba4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:18.714978 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40f460-0370-46b4-b7b8-4b0b8fbcdba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:18.732279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-3c40f460-0370-46b4-b7b8-4b0b8fbcdba4 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:18.733389 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3c40f460-0370-46b4-b7b8-4b0b8fbcdba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:18.769562 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2748ef-5c09-4afb-a8eb-7495200c977b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:18.783935 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:18.805279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 372/734] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:21:18.805662 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06940eac-b86a-4701-92d7-cfef1975e311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:18.806348 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40f460-0370-46b4-b7b8-4b0b8fbcdba4 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:18.806865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06940eac-b86a-4701-92d7-cfef1975e311 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:18.807368 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06940eac-b86a-4701-92d7-cfef1975e311 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:18.807817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06940eac-b86a-4701-92d7-cfef1975e311 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:18.824238 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06940eac-b86a-4701-92d7-cfef1975e311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:18.826532 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06940eac-b86a-4701-92d7-cfef1975e311 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:18.844614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-06940eac-b86a-4701-92d7-cfef1975e311 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:18.844614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:18.844614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:18.844614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:18.844614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:18.844614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:18.844614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:18.844614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:18.844614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:18.844614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:18.844614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:18.844614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:18.844614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:18.844614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:18.844614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:18.845544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:18.845544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:18.845544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:18.845544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:18.845544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:18.845544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:18.845544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:18.845544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:18.845544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:18.845544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:18.845544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:18.845544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:18.845544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:18.845544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:18.845544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:18.845544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:18.845544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:18.846419 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:18.846419 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:18.846419 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:18.846419 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:18.848053 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 363/735] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:18.848960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06940eac-b86a-4701-92d7-cfef1975e311 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:18.904847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:18.927244 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40f460-0370-46b4-b7b8-4b0b8fbcdba4 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:18.964811 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06940eac-b86a-4701-92d7-cfef1975e311 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:18.967996 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc01c3-0703-4f46-b70a-049d1bbf8a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:18.968801 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc01c3-0703-4f46-b70a-049d1bbf8a6b None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:18.969080 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc01c3-0703-4f46-b70a-049d1bbf8a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:18.969370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc01c3-0703-4f46-b70a-049d1bbf8a6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:18.979541 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc01c3-0703-4f46-b70a-049d1bbf8a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:18.980969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc01c3-0703-4f46-b70a-049d1bbf8a6b None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:19.001226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-b4cc01c3-0703-4f46-b70a-049d1bbf8a6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:19.001226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:19.001226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:19.001226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:19.001226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:19.001226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:19.001226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:19.001226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:19.001226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:19.001226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:19.001226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:19.001226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:19.001226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:19.001226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:19.001226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:19.002184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:19.002184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:19.002184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:19.002184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:19.002184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:19.002184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:19.002184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:19.002184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:19.002184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:19.002184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:19.002184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:19.002184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:19.002184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:19.002184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:19.002184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:19.002184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:19.002184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:19.003089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:19.003089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:19.003089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:19.003089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:19.003327 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d05da-20a8-488f-bf08-9bd73b0fd53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:19.003327 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d05da-20a8-488f-bf08-9bd73b0fd53b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:19.003472 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 373/736] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:19.004082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc01c3-0703-4f46-b70a-049d1bbf8a6b None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:19.007007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d05da-20a8-488f-bf08-9bd73b0fd53b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:19.008049 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d05da-20a8-488f-bf08-9bd73b0fd53b None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:19.008049 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d05da-20a8-488f-bf08-9bd73b0fd53b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:19.008308 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d05da-20a8-488f-bf08-9bd73b0fd53b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:19.009646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d05da-20a8-488f-bf08-9bd73b0fd53b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:19.028440 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 364/737] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:19.028440 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d05da-20a8-488f-bf08-9bd73b0fd53b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:19.040161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:19.117184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc01c3-0703-4f46-b70a-049d1bbf8a6b None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:19.150571 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d05da-20a8-488f-bf08-9bd73b0fd53b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:19.152712 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edec6e7-80e8-42b5-967b-0983954fb9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:19.153180 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:19.154620 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edec6e7-80e8-42b5-967b-0983954fb9b3 None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:19.156778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edec6e7-80e8-42b5-967b-0983954fb9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:19.157230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edec6e7-80e8-42b5-967b-0983954fb9b3 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:19.158321 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edec6e7-80e8-42b5-967b-0983954fb9b3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:19.160284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edec6e7-80e8-42b5-967b-0983954fb9b3 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:19.177205 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 374/738] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 14 20:21:19 2022] POST /identity/v3/services => generated 251 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:21:19.178035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edec6e7-80e8-42b5-967b-0983954fb9b3 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:19.208761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e75a80-8048-4590-bfe4-e0582455698b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:19.210639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e75a80-8048-4590-bfe4-e0582455698b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:19.210925 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e75a80-8048-4590-bfe4-e0582455698b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:19.211223 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e75a80-8048-4590-bfe4-e0582455698b None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:19.211969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e75a80-8048-4590-bfe4-e0582455698b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:19.212689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e75a80-8048-4590-bfe4-e0582455698b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:19.214553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e75a80-8048-4590-bfe4-e0582455698b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:19.228769 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 365/739] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:19.230548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e75a80-8048-4590-bfe4-e0582455698b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:19.293058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:19.302330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edec6e7-80e8-42b5-967b-0983954fb9b3 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:19.342978 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db49f9a-8c64-408f-b2e2-335a30f8ea0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:19.343686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db49f9a-8c64-408f-b2e2-335a30f8ea0c None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:19.343980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db49f9a-8c64-408f-b2e2-335a30f8ea0c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:19.344256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db49f9a-8c64-408f-b2e2-335a30f8ea0c None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:19.344985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db49f9a-8c64-408f-b2e2-335a30f8ea0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:19.345290 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db49f9a-8c64-408f-b2e2-335a30f8ea0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:19.346675 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db49f9a-8c64-408f-b2e2-335a30f8ea0c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:19.352398 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e75a80-8048-4590-bfe4-e0582455698b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:19.359544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 375/740] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:19.361345 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7141b8-4de1-4003-9db8-128f0ee45694 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:19.361345 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7141b8-4de1-4003-9db8-128f0ee45694 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:19.361535 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7141b8-4de1-4003-9db8-128f0ee45694 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:19.363030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 376/741] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:19.363750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7141b8-4de1-4003-9db8-128f0ee45694 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:19.397288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ee03c-8c2f-4a90-85ce-0c2258e53206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:19.398454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ee03c-8c2f-4a90-85ce-0c2258e53206 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:19.399069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ee03c-8c2f-4a90-85ce-0c2258e53206 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:19.399652 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ee03c-8c2f-4a90-85ce-0c2258e53206 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:19.408477 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ee03c-8c2f-4a90-85ce-0c2258e53206 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:19.410326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ee03c-8c2f-4a90-85ce-0c2258e53206 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:19.425893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-025ee03c-8c2f-4a90-85ce-0c2258e53206 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 20:21:19.425893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:19.425893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:19.425893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:19.425893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:19.425893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:19.425893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:19.425893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:19.425893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:19.425893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:19.425893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:19.425893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:19.425893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:19.425893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:19.425893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:19.425893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:19.426831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:19.426831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:19.426831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:19.426831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:19.426831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:19.426831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:19.426831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:19.426831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:19.426831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:19.426831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:19.426831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:19.426831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:19.426831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:19.426831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:19.426831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:19.426831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:19.426831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:19.426831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:19.428085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:19.428085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:19.428085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:19.428085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:19.428085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:19.428085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:19.428085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:19.428085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:19.428085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 20:21:19.428085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:19.428610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 366/742] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:21:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:19.429153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ee03c-8c2f-4a90-85ce-0c2258e53206 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:19.456903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:19.476490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7141b8-4de1-4003-9db8-128f0ee45694 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:19.513400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581b9d1-ffa7-41ce-a745-f15466af38e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:19.514229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581b9d1-ffa7-41ce-a745-f15466af38e1 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:19.514511 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581b9d1-ffa7-41ce-a745-f15466af38e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:19.514795 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581b9d1-ffa7-41ce-a745-f15466af38e1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:19.522562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a581b9d1-ffa7-41ce-a745-f15466af38e1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:19.524059 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a581b9d1-ffa7-41ce-a745-f15466af38e1 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:19.540338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-a581b9d1-ffa7-41ce-a745-f15466af38e1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 20:21:19.540338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:19.540338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:19.540338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:19.540338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:19.540338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:19.540338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:19.540338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:19.540338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:19.540338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:19.540338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:19.540338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:19.540338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:19.540338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:19.540338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:19.540338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:19.541693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:19.541693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:19.541693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:19.541693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:19.541693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:19.541693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:19.541693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:19.541693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:19.541693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:19.541693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:19.541693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:19.541693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:19.541693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:19.541693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:19.541693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:19.541693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:19.541693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:19.541693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:19.543202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:19.543202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:19.543202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:19.543202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:19.543202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:19.543202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:19.543202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:19.543202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:19.543202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 20:21:19.543202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:19.543202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 377/743] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 14 20:21:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:19.543202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde9b0c-a2e5-4f3d-b2ce-36c635b0f0c4 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:19.543202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde9b0c-a2e5-4f3d-b2ce-36c635b0f0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:19.543918 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde9b0c-a2e5-4f3d-b2ce-36c635b0f0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:19.557037 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ee03c-8c2f-4a90-85ce-0c2258e53206 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:19.562293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-ccde9b0c-a2e5-4f3d-b2ce-36c635b0f0c4 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:19.563511 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ccde9b0c-a2e5-4f3d-b2ce-36c635b0f0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:19.572864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:19.596367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1712e-c6de-4afd-b42c-86c753080795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:19.597565 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1712e-c6de-4afd-b42c-86c753080795 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:19.598187 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1712e-c6de-4afd-b42c-86c753080795 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:19.598685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1712e-c6de-4afd-b42c-86c753080795 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:19.613243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1712e-c6de-4afd-b42c-86c753080795 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:19.614900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1712e-c6de-4afd-b42c-86c753080795 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:19.629654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-0cd1712e-c6de-4afd-b42c-86c753080795 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 20:21:19.629654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:19.629654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:19.629654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:19.629654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:19.629654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:19.629654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:19.629654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:19.629654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:19.629654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:19.629654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:19.629654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:19.629654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:19.629654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:19.629654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:19.629654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:19.630610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:19.630610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:19.630610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:19.630610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:19.630610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:19.630610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:19.630610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:19.630610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:19.630610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:19.630610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:19.630610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:19.630610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:19.630610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:19.630610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:19.630610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:19.630610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:19.630610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:19.630610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:19.631996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:19.631996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:19.631996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:19.631996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:19.631996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:19.631996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:19.631996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:19.631996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:19.631996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 20:21:19.631996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:19.631996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 367/744] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 14 20:21:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:19.632589 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1712e-c6de-4afd-b42c-86c753080795 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:19.655109 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 378/745] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 20:21:19.655109 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde9b0c-a2e5-4f3d-b2ce-36c635b0f0c4 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:19.744596 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:19.766410 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde9b0c-a2e5-4f3d-b2ce-36c635b0f0c4 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:19.767654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1712e-c6de-4afd-b42c-86c753080795 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:19.807682 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c39fd-f2f2-4f36-b1f6-492491c1da8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:19.808318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c39fd-f2f2-4f36-b1f6-492491c1da8d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:19.808678 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c39fd-f2f2-4f36-b1f6-492491c1da8d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:19.809014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c39fd-f2f2-4f36-b1f6-492491c1da8d None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:19.810158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663ccd6-b1a4-413d-b439-4bb369adc153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:19.810508 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c39fd-f2f2-4f36-b1f6-492491c1da8d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:19.810827 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663ccd6-b1a4-413d-b439-4bb369adc153 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:19.810970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c39fd-f2f2-4f36-b1f6-492491c1da8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:19.811138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663ccd6-b1a4-413d-b439-4bb369adc153 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:19.811400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663ccd6-b1a4-413d-b439-4bb369adc153 None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:19.812241 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663ccd6-b1a4-413d-b439-4bb369adc153 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:19.812349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c39fd-f2f2-4f36-b1f6-492491c1da8d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:19.812553 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663ccd6-b1a4-413d-b439-4bb369adc153 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:19.813965 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663ccd6-b1a4-413d-b439-4bb369adc153 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:19.836765 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 379/746] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jul 14 20:21:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:19.836765 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663ccd6-b1a4-413d-b439-4bb369adc153 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:19.837034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 368/747] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jul 14 20:21:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:19.837416 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c39fd-f2f2-4f36-b1f6-492491c1da8d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:19.839361 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:19.943717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663ccd6-b1a4-413d-b439-4bb369adc153 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:19.947456 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c39fd-f2f2-4f36-b1f6-492491c1da8d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:19.986265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47a3e3-922d-4dd3-8a5e-f4aa0ef05f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:19.986959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47a3e3-922d-4dd3-8a5e-f4aa0ef05f1b None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:19.987241 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47a3e3-922d-4dd3-8a5e-f4aa0ef05f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:19.987518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47a3e3-922d-4dd3-8a5e-f4aa0ef05f1b None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:19.988356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47a3e3-922d-4dd3-8a5e-f4aa0ef05f1b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:19.988667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47a3e3-922d-4dd3-8a5e-f4aa0ef05f1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:19.990101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47a3e3-922d-4dd3-8a5e-f4aa0ef05f1b None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:19.990724 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc53f36-087d-48e7-ba28-4796be989f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:19.991440 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc53f36-087d-48e7-ba28-4796be989f28 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:19.991727 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc53f36-087d-48e7-ba28-4796be989f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:19.991996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc53f36-087d-48e7-ba28-4796be989f28 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:19.992834 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc53f36-087d-48e7-ba28-4796be989f28 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:19.994276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc53f36-087d-48e7-ba28-4796be989f28 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:20.004549 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:20.010691 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 380/748] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 14 20:21:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:20.011742 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47a3e3-922d-4dd3-8a5e-f4aa0ef05f1b None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:20.019436 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 369/749] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 14 20:21:19 2022] POST /identity/v3/services => generated 240 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:21:20.020274 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc53f36-087d-48e7-ba28-4796be989f28 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:20.090138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:20.142885 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47a3e3-922d-4dd3-8a5e-f4aa0ef05f1b None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:20.179274 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc53f36-087d-48e7-ba28-4796be989f28 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:20.184133 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b51ba4-ef08-489b-bb87-976d9ebbad2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:20.185366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b51ba4-ef08-489b-bb87-976d9ebbad2c None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:20.185651 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b51ba4-ef08-489b-bb87-976d9ebbad2c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:20.185925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b51ba4-ef08-489b-bb87-976d9ebbad2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:20.229145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6703cd-a404-4436-b80f-a776009405c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:20.229802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6703cd-a404-4436-b80f-a776009405c5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:20.230083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6703cd-a404-4436-b80f-a776009405c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:20.230355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6703cd-a404-4436-b80f-a776009405c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:20.272122 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b51ba4-ef08-489b-bb87-976d9ebbad2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:20.274064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b51ba4-ef08-489b-bb87-976d9ebbad2c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:20.296741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:20.304665 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6703cd-a404-4436-b80f-a776009405c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:20.306102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6703cd-a404-4436-b80f-a776009405c5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:20.346470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:20.394751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 381/750] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:21:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:21:20.396055 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b51ba4-ef08-489b-bb87-976d9ebbad2c None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:20.420709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 370/751] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:21:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:21:20.421677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6703cd-a404-4436-b80f-a776009405c5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:20.506454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b51ba4-ef08-489b-bb87-976d9ebbad2c None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:20.526330 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:20.533541 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6703cd-a404-4436-b80f-a776009405c5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:20.547287 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59bc83-1b0f-495c-8208-49f3fcdb1b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:20.548531 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59bc83-1b0f-495c-8208-49f3fcdb1b3f None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:20.549121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59bc83-1b0f-495c-8208-49f3fcdb1b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:20.549595 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59bc83-1b0f-495c-8208-49f3fcdb1b3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:20.561527 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59bc83-1b0f-495c-8208-49f3fcdb1b3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:20.563385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59bc83-1b0f-495c-8208-49f3fcdb1b3f None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:20.570171 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2b373-0ba3-4ef4-bdf8-e4f507b3a9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:20.570977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2b373-0ba3-4ef4-bdf8-e4f507b3a9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:20.571265 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2b373-0ba3-4ef4-bdf8-e4f507b3a9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:20.571541 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2b373-0ba3-4ef4-bdf8-e4f507b3a9b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:20.582553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2b373-0ba3-4ef4-bdf8-e4f507b3a9b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:20.584055 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2b373-0ba3-4ef4-bdf8-e4f507b3a9b0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:20.584394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-6f59bc83-1b0f-495c-8208-49f3fcdb1b3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:20.584394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:20.584394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:20.584394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:20.584394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:20.584394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:20.584394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:20.584394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:20.584394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:20.584394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:20.584394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:20.584394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:20.584394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:20.584394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:20.584394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:20.585267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:20.585267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:20.585267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:20.585267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:20.585267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:20.585267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:20.585267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:20.585267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:20.585267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:20.585267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:20.585267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:20.585267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:20.585267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:20.585267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:20.585267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:20.585267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:20.585267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:20.586186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:20.586186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:20.586186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:20.586186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:20.588513 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 382/752] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:20.589506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59bc83-1b0f-495c-8208-49f3fcdb1b3f None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:20.603262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-70a2b373-0ba3-4ef4-bdf8-e4f507b3a9b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:20.603262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:20.603262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:20.603262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:20.603262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:20.603262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:20.603262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:20.603262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:20.603262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:20.603262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:20.603262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:20.603262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:20.603262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:20.603262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:20.603262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:20.604176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:20.604176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:20.604176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:20.604176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:20.604176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:20.604176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:20.604176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:20.604176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:20.604176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:20.604176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:20.604176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:20.604176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:20.604176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:20.604176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:20.604176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:20.604176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:20.604176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:20.605108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:20.605108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:20.605108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:20.605108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:20.605108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 371/753] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:20.605814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2b373-0ba3-4ef4-bdf8-e4f507b3a9b0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:20.607958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:20.724965 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59bc83-1b0f-495c-8208-49f3fcdb1b3f None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:20.738753 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2b373-0ba3-4ef4-bdf8-e4f507b3a9b0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:20.769148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3fc18-0752-4a9a-a43f-dcc54f4a1ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:20.770012 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3fc18-0752-4a9a-a43f-dcc54f4a1ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:20.770298 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3fc18-0752-4a9a-a43f-dcc54f4a1ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:20.770571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3fc18-0752-4a9a-a43f-dcc54f4a1ce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:20.782309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3fc18-0752-4a9a-a43f-dcc54f4a1ce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:20.784037 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3fc18-0752-4a9a-a43f-dcc54f4a1ce1 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:20.789967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d51016-5b77-4cc2-b838-7f857075025a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:20.790961 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d51016-5b77-4cc2-b838-7f857075025a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:20.791448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d51016-5b77-4cc2-b838-7f857075025a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:20.791947 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d51016-5b77-4cc2-b838-7f857075025a None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:20.793539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d51016-5b77-4cc2-b838-7f857075025a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:20.794595 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d51016-5b77-4cc2-b838-7f857075025a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:20.796432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d51016-5b77-4cc2-b838-7f857075025a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:20.807994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-83c3fc18-0752-4a9a-a43f-dcc54f4a1ce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:20.807994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:20.807994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:20.807994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:20.807994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:20.807994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:20.807994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:20.807994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:20.807994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:20.807994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:20.807994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:20.807994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:20.807994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:20.807994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:20.807994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:20.808947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:20.808947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:20.808947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:20.808947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:20.808947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:20.808947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:20.808947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:20.808947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:20.808947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:20.808947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:20.808947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:20.808947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:20.808947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:20.808947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:20.808947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:20.808947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:20.808947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:20.810412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:20.810412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:20.810412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:20.810412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:20.813760 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 372/754] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:20.814637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 383/755] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:20.815463 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d51016-5b77-4cc2-b838-7f857075025a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:20.815674 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3fc18-0752-4a9a-a43f-dcc54f4a1ce1 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:20.827191 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:20.884634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:20.929550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3fc18-0752-4a9a-a43f-dcc54f4a1ce1 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:20.958517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d51016-5b77-4cc2-b838-7f857075025a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:20.966131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336d107-ca48-48ae-809e-57f301647c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:20.966906 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336d107-ca48-48ae-809e-57f301647c56 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:20.967311 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336d107-ca48-48ae-809e-57f301647c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:20.967684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336d107-ca48-48ae-809e-57f301647c56 None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:20.968728 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336d107-ca48-48ae-809e-57f301647c56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:20.969086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336d107-ca48-48ae-809e-57f301647c56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:20.970537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336d107-ca48-48ae-809e-57f301647c56 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:20.987738 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 384/756] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:20.988519 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336d107-ca48-48ae-809e-57f301647c56 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:21.016275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0acaa7-7009-4913-8a35-778ff07d5976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:21.017207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0acaa7-7009-4913-8a35-778ff07d5976 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:21.017797 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0acaa7-7009-4913-8a35-778ff07d5976 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:21.018363 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0acaa7-7009-4913-8a35-778ff07d5976 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:21.019783 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0acaa7-7009-4913-8a35-778ff07d5976 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:21.020248 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0acaa7-7009-4913-8a35-778ff07d5976 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:21.022528 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0acaa7-7009-4913-8a35-778ff07d5976 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:21.041286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 373/757] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:21.042553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0acaa7-7009-4913-8a35-778ff07d5976 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:21.111148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:21.124012 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336d107-ca48-48ae-809e-57f301647c56 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:21.149005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0acaa7-7009-4913-8a35-778ff07d5976 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:21.155728 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:21.171016 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2498830-c1ed-4a7c-904d-9dbc5229da79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:21.172098 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2498830-c1ed-4a7c-904d-9dbc5229da79 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:21.172955 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2498830-c1ed-4a7c-904d-9dbc5229da79 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:21.173381 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2498830-c1ed-4a7c-904d-9dbc5229da79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:21.183698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2498830-c1ed-4a7c-904d-9dbc5229da79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:21.185853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2498830-c1ed-4a7c-904d-9dbc5229da79 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:21.186775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0c727-467a-488a-80e8-eb567a52c8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:21.187705 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0c727-467a-488a-80e8-eb567a52c8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:21.187997 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0c727-467a-488a-80e8-eb567a52c8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:21.188302 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0c727-467a-488a-80e8-eb567a52c8b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:21.195789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a0c727-467a-488a-80e8-eb567a52c8b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:21.197230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a0c727-467a-488a-80e8-eb567a52c8b7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:21.204815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-f2498830-c1ed-4a7c-904d-9dbc5229da79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:21.204815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:21.204815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:21.204815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:21.204815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:21.204815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:21.204815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:21.204815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:21.204815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:21.204815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:21.204815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:21.204815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:21.204815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:21.204815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:21.204815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:21.205923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:21.205923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:21.205923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:21.205923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:21.205923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:21.205923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:21.205923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:21.205923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:21.205923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:21.205923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:21.205923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:21.205923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:21.205923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:21.205923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:21.205923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:21.205923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:21.205923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:21.206826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:21.206826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:21.206826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:21.206826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:21.206826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:21.206826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:21.206826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:21.206826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:21.209443 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 385/758] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:21.210545 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2498830-c1ed-4a7c-904d-9dbc5229da79 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:21.221133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-d7a0c727-467a-488a-80e8-eb567a52c8b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:21.221133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:21.221133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:21.221133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:21.221133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:21.221133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:21.221133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:21.221133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:21.221133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:21.221133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:21.221133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:21.221133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:21.221133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:21.221133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:21.221133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:21.222081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:21.222081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:21.222081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:21.222081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:21.222081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:21.222081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:21.222081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:21.222081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:21.222081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:21.222081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:21.222081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:21.222081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:21.222081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:21.222081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:21.222081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:21.222081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:21.222081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:21.222977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:21.222977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:21.222977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:21.222977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:21.222977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:21.222977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:21.222977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:21.222977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:21.224622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 374/759] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:21.225588 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0c727-467a-488a-80e8-eb567a52c8b7 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:21.320923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2498830-c1ed-4a7c-904d-9dbc5229da79 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:21.337560 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0c727-467a-488a-80e8-eb567a52c8b7 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:21.359177 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a079085-088c-4d98-b730-4e6b2574d294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:21.360191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a079085-088c-4d98-b730-4e6b2574d294 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:21.360650 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a079085-088c-4d98-b730-4e6b2574d294 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:21.361179 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a079085-088c-4d98-b730-4e6b2574d294 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:21.364057 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:21.369817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a079085-088c-4d98-b730-4e6b2574d294 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:21.371294 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a079085-088c-4d98-b730-4e6b2574d294 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:21.374372 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d057e4c-1eb3-4b65-a0c1-ee726d072bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:21.375253 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d057e4c-1eb3-4b65-a0c1-ee726d072bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:21.375732 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d057e4c-1eb3-4b65-a0c1-ee726d072bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:21.376152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d057e4c-1eb3-4b65-a0c1-ee726d072bb0 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:21.377358 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d057e4c-1eb3-4b65-a0c1-ee726d072bb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:21.377814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d057e4c-1eb3-4b65-a0c1-ee726d072bb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:21.379521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d057e4c-1eb3-4b65-a0c1-ee726d072bb0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:21.384944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-3a079085-088c-4d98-b730-4e6b2574d294 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:21.384944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:21.384944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:21.384944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:21.384944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:21.384944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:21.384944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:21.384944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:21.384944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:21.384944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:21.384944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:21.384944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:21.384944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:21.384944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:21.384944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:21.385982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:21.385982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:21.385982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:21.385982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:21.385982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:21.385982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:21.385982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:21.385982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:21.385982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:21.385982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:21.385982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:21.385982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:21.385982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:21.385982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:21.385982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:21.385982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:21.385982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:21.386959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:21.386959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:21.386959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:21.386959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:21.386959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:21.386959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:21.386959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:21.386959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:21.386959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 386/760] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:21.387467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a079085-088c-4d98-b730-4e6b2574d294 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:21.402881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 375/761] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:21.402881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b1c7c-ecbb-4038-b706-ddc5d24aa813 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:21.402881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b1c7c-ecbb-4038-b706-ddc5d24aa813 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:21.402881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b1c7c-ecbb-4038-b706-ddc5d24aa813 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:21.402881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 376/762] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:21.402881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b1c7c-ecbb-4038-b706-ddc5d24aa813 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:21.440375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:21.500005 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a079085-088c-4d98-b730-4e6b2574d294 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:21.515948 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b1c7c-ecbb-4038-b706-ddc5d24aa813 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:21.541182 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed0a7a-8e2b-4ec6-93c2-3618593b8429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:21.541853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed0a7a-8e2b-4ec6-93c2-3618593b8429 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:21.542138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed0a7a-8e2b-4ec6-93c2-3618593b8429 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:21.542414 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed0a7a-8e2b-4ec6-93c2-3618593b8429 None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:21.543247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed0a7a-8e2b-4ec6-93c2-3618593b8429 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:21.543566 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed0a7a-8e2b-4ec6-93c2-3618593b8429 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:21.545038 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed0a7a-8e2b-4ec6-93c2-3618593b8429 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:21.556057 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6ef54-27f1-40bf-88d5-4691874eac6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:21.556728 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6ef54-27f1-40bf-88d5-4691874eac6c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:21.557012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6ef54-27f1-40bf-88d5-4691874eac6c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:21.557294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6ef54-27f1-40bf-88d5-4691874eac6c None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:21.558083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6ef54-27f1-40bf-88d5-4691874eac6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:21.558473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6ef54-27f1-40bf-88d5-4691874eac6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:21.559964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6ef54-27f1-40bf-88d5-4691874eac6c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:21.562501 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 387/763] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:21.563250 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed0a7a-8e2b-4ec6-93c2-3618593b8429 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:21.572443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 377/764] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:21.573750 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e19fc-68ec-4a4e-8af0-2515c8e167ce None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:21.574044 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e19fc-68ec-4a4e-8af0-2515c8e167ce None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:21.574323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e19fc-68ec-4a4e-8af0-2515c8e167ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:21.690901 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed0a7a-8e2b-4ec6-93c2-3618593b8429 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:21.694621 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:21.706766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ef2e19fc-68ec-4a4e-8af0-2515c8e167ce None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:21.708010 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ef2e19fc-68ec-4a4e-8af0-2515c8e167ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:21.732609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7baaae-7e9f-4871-83db-aa8a92ff0b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:21.733939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7baaae-7e9f-4871-83db-aa8a92ff0b32 None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:21.734383 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7baaae-7e9f-4871-83db-aa8a92ff0b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:21.734645 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7baaae-7e9f-4871-83db-aa8a92ff0b32 None admin] PATH_INFO: `/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/aa3bb9f1e24a4e9ebd460e97b3736bc0/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:21.736690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7baaae-7e9f-4871-83db-aa8a92ff0b32 None admin] RBAC: Authorizing `identity:create_grant(project_id=f33c8ac5154f46a89d1a0dc11fb58cf4, user_id=aa3bb9f1e24a4e9ebd460e97b3736bc0, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:21.738126 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7baaae-7e9f-4871-83db-aa8a92ff0b32 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:21.752696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 388/765] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:21:21 2022] PUT /identity/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/aa3bb9f1e24a4e9ebd460e97b3736bc0/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:21:21.753523 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7baaae-7e9f-4871-83db-aa8a92ff0b32 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:21.760800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:21.789494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 378/766] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:21.790775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e19fc-68ec-4a4e-8af0-2515c8e167ce None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:21.926675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e19fc-68ec-4a4e-8af0-2515c8e167ce None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:21.935808 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7baaae-7e9f-4871-83db-aa8a92ff0b32 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:21.976916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe600eb-2d8d-44ab-8b0f-fbd62918f00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:21.977605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe600eb-2d8d-44ab-8b0f-fbd62918f00f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:21.977894 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe600eb-2d8d-44ab-8b0f-fbd62918f00f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:21.978176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe600eb-2d8d-44ab-8b0f-fbd62918f00f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:21.979150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe600eb-2d8d-44ab-8b0f-fbd62918f00f None admin] RBAC: Adding query filter params (role.id=5ca112885467422684e134cb0f94865a, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=acc23233929144a5b7b3a1733da19828) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:21.979560 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe600eb-2d8d-44ab-8b0f-fbd62918f00f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:21.980434 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2240a-9c22-41ba-8517-d76cd7a2faf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:21.981061 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe600eb-2d8d-44ab-8b0f-fbd62918f00f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:21.981170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2240a-9c22-41ba-8517-d76cd7a2faf2 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:21.981365 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2240a-9c22-41ba-8517-d76cd7a2faf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:21.981693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2240a-9c22-41ba-8517-d76cd7a2faf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:21.982523 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2240a-9c22-41ba-8517-d76cd7a2faf2 None admin] RBAC: Adding query filter params (role.id=ea49c36b4a7f4cfb9048065638d458af, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=4f75b8b484384554ad59aa20535875e9) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:21.983145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2240a-9c22-41ba-8517-d76cd7a2faf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:21.985206 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2240a-9c22-41ba-8517-d76cd7a2faf2 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:21.987432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 379/767] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:21:21 2022] GET /identity/v3/role_assignments?user.id=acc23233929144a5b7b3a1733da19828&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=5ca112885467422684e134cb0f94865a => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:21.988492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe600eb-2d8d-44ab-8b0f-fbd62918f00f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:21.999303 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 389/768] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:21:21 2022] GET /identity/v3/role_assignments?user.id=4f75b8b484384554ad59aa20535875e9&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=ea49c36b4a7f4cfb9048065638d458af => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:22.050060 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:22.118765 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe600eb-2d8d-44ab-8b0f-fbd62918f00f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:22.155680 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660fed2-b2a4-4cf5-87c5-19f43e2d3f18 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:22.156017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660fed2-b2a4-4cf5-87c5-19f43e2d3f18 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:22.156236 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660fed2-b2a4-4cf5-87c5-19f43e2d3f18 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:22.160485 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e7719-ce96-4295-964b-c9257ba119b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:22.161317 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e7719-ce96-4295-964b-c9257ba119b8 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:22.161599 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e7719-ce96-4295-964b-c9257ba119b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:22.161719 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 390/769] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:22.162145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e7719-ce96-4295-964b-c9257ba119b8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:22.162535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e7719-ce96-4295-964b-c9257ba119b8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:22.163600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e242b-1bb0-4e8b-9c01-7d48dc77ad49 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:22.163888 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e242b-1bb0-4e8b-9c01-7d48dc77ad49 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:22.163990 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e7719-ce96-4295-964b-c9257ba119b8 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:22.164178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e242b-1bb0-4e8b-9c01-7d48dc77ad49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:22.179700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-5f5e7719-ce96-4295-964b-c9257ba119b8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 20:21:22.179700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:22.179700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:22.179700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:22.179700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:22.179700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:22.179700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:22.179700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:22.179700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 20:21:22.179700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:22.179700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:22.179700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:22.179700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:22.179700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:22.179700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:22.179700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:22.180695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:22.180695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:22.180695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:22.180695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:22.180695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:22.180695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:22.180695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:22.180695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:22.180695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:22.180695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:22.180695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:22.180695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:22.180695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:22.180695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:22.180695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:22.180695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:22.181530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:22.181530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:22.181530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:22.181530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:22.181530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:22.181530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:22.181530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:22.181530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:22.181530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:22.181530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:22.181530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:22.181530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:22.181530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:22.181530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 20:21:22.181530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:22.181530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 380/770] 10.222.0.52 () {56 vars in 1129 bytes} [Thu Jul 14 20:21:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:22.188143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-8b1e242b-1bb0-4e8b-9c01-7d48dc77ad49 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:22.189397 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-8b1e242b-1bb0-4e8b-9c01-7d48dc77ad49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:22.193235 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e7719-ce96-4295-964b-c9257ba119b8 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:22.231428 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:22.306037 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 391/771] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 20:21:22.345192 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e7719-ce96-4295-964b-c9257ba119b8 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:22.347089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e242b-1bb0-4e8b-9c01-7d48dc77ad49 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:22.354744 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:22.383579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfc71b-d6a4-4c82-881b-6c10087a97ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:22.384636 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfc71b-d6a4-4c82-881b-6c10087a97ee None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:22.385175 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfc71b-d6a4-4c82-881b-6c10087a97ee None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:22.385659 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfc71b-d6a4-4c82-881b-6c10087a97ee None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:22.386630 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfc71b-d6a4-4c82-881b-6c10087a97ee None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:22.387166 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfc71b-d6a4-4c82-881b-6c10087a97ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:22.389319 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfc71b-d6a4-4c82-881b-6c10087a97ee None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:22.399073 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 381/772] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jul 14 20:21:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:22.401440 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfc71b-d6a4-4c82-881b-6c10087a97ee None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:22.507365 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfc71b-d6a4-4c82-881b-6c10087a97ee None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:22.544214 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a02629-df9a-4e69-b767-6d2b1ceeb6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:22.545480 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a02629-df9a-4e69-b767-6d2b1ceeb6ba None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:22.545957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a02629-df9a-4e69-b767-6d2b1ceeb6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:22.546434 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a02629-df9a-4e69-b767-6d2b1ceeb6ba None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:22.547392 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a02629-df9a-4e69-b767-6d2b1ceeb6ba None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:22.547895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a02629-df9a-4e69-b767-6d2b1ceeb6ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:22.549491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a02629-df9a-4e69-b767-6d2b1ceeb6ba None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:22.560822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 382/773] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jul 14 20:21:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:22.562394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a02629-df9a-4e69-b767-6d2b1ceeb6ba None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:22.599420 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:22.686856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e242b-1bb0-4e8b-9c01-7d48dc77ad49 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:22.714345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a02629-df9a-4e69-b767-6d2b1ceeb6ba None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:22.751328 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aeadfb-a5c9-42f8-8b76-f84ef0ca7afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:22.752511 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aeadfb-a5c9-42f8-8b76-f84ef0ca7afc None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:22.753944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aeadfb-a5c9-42f8-8b76-f84ef0ca7afc None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:22.755135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aeadfb-a5c9-42f8-8b76-f84ef0ca7afc None admin] PATH_INFO: `/v3/services/network` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:22.756675 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aeadfb-a5c9-42f8-8b76-f84ef0ca7afc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:22.758325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aeadfb-a5c9-42f8-8b76-f84ef0ca7afc None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:22.776251 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006ca78-4d8d-45b1-8763-a7529399ec11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:22.777258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006ca78-4d8d-45b1-8763-a7529399ec11 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:22.777735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006ca78-4d8d-45b1-8763-a7529399ec11 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:22.778203 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006ca78-4d8d-45b1-8763-a7529399ec11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:22.779371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6006ca78-4d8d-45b1-8763-a7529399ec11 None admin] RBAC: Adding query filter params (interface=public, service_id=20ec4619c89c47f4bfc3938408a0d56f, region_id=RegionOne) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:22.783226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-d7aeadfb-a5c9-42f8-8b76-f84ef0ca7afc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 20:21:22.783226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:22.783226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:22.783226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:22.783226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:22.783226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:22.783226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:22.783226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:22.783226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 20:21:22.783226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:22.783226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:22.783226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:22.783226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:22.783226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:22.783226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:22.783226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:22.783226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:22.784187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:22.784187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:22.784187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:22.784187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:22.784187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:22.784187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:22.784187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:22.784187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:22.784187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:22.784187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:22.784187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:22.784187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:22.784187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:22.784187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:22.784187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:22.784187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:22.785036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:22.785036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:22.785036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:22.785036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:22.785036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:22.785036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:22.785036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:22.785036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:22.785036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:22.785036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:22.785036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:22.785036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:22.785036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 20:21:22.785036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:22.786590 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6006ca78-4d8d-45b1-8763-a7529399ec11 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:22.788200 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6006ca78-4d8d-45b1-8763-a7529399ec11 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:22.793559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 392/774] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:22 2022] GET /identity/v3/services/network => generated 88 bytes in 447 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:21:22.803098 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aeadfb-a5c9-42f8-8b76-f84ef0ca7afc None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:22.806399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 383/775] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 14 20:21:22 2022] GET /identity/v3/endpoints?service_id=20ec4619c89c47f4bfc3938408a0d56f&interface=public®ion_id=RegionOne => generated 194 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:22.808192 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006ca78-4d8d-45b1-8763-a7529399ec11 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:22.849776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:22.938805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:22.952353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006ca78-4d8d-45b1-8763-a7529399ec11 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:23.012023 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aeadfb-a5c9-42f8-8b76-f84ef0ca7afc None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:23.023727 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402607fd-c36b-4fa0-8a21-56be19991b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:23.024720 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402607fd-c36b-4fa0-8a21-56be19991b7a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:23.025200 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402607fd-c36b-4fa0-8a21-56be19991b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:23.025709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402607fd-c36b-4fa0-8a21-56be19991b7a None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:23.026684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402607fd-c36b-4fa0-8a21-56be19991b7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:23.028267 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402607fd-c36b-4fa0-8a21-56be19991b7a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:23.034313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 384/776] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 14 20:21:22 2022] GET /identity/v3/services => generated 772 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:23.067434 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ad748-03b8-49c9-8ce1-51f58fe388d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:23.068129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ad748-03b8-49c9-8ce1-51f58fe388d7 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:23.068413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ad748-03b8-49c9-8ce1-51f58fe388d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:23.068690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ad748-03b8-49c9-8ce1-51f58fe388d7 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:23.069530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ad748-03b8-49c9-8ce1-51f58fe388d7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:23.069893 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ad748-03b8-49c9-8ce1-51f58fe388d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:23.071320 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ad748-03b8-49c9-8ce1-51f58fe388d7 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:23.081568 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 393/777] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 14 20:21:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:23.082969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402607fd-c36b-4fa0-8a21-56be19991b7a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:23.222382 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402607fd-c36b-4fa0-8a21-56be19991b7a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:23.261683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919bd9c-95ec-409e-8aac-9cb64e765df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:23.262408 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919bd9c-95ec-409e-8aac-9cb64e765df9 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:23.262641 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919bd9c-95ec-409e-8aac-9cb64e765df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:23.262960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919bd9c-95ec-409e-8aac-9cb64e765df9 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:23.263692 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e919bd9c-95ec-409e-8aac-9cb64e765df9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:23.264004 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e919bd9c-95ec-409e-8aac-9cb64e765df9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:23.265382 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e919bd9c-95ec-409e-8aac-9cb64e765df9 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:23.271275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 385/778] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 14 20:21:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:23.277076 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ad748-03b8-49c9-8ce1-51f58fe388d7 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:23.372700 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:23.391995 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ad748-03b8-49c9-8ce1-51f58fe388d7 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:23.438976 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb153d9-7b5c-42a0-aaca-5003c85b2ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:23.439660 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb153d9-7b5c-42a0-aaca-5003c85b2ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:23.439946 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb153d9-7b5c-42a0-aaca-5003c85b2ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:23.440227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb153d9-7b5c-42a0-aaca-5003c85b2ef9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:23.441032 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb153d9-7b5c-42a0-aaca-5003c85b2ef9 None admin] RBAC: Adding query filter params (interface=public, service_id=f48e995cfe144c4c8dd2935a51ec8201, region_id=RegionOne) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:23.441345 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb153d9-7b5c-42a0-aaca-5003c85b2ef9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:23.443020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb153d9-7b5c-42a0-aaca-5003c85b2ef9 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:23.459092 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 394/779] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 14 20:21:23 2022] GET /identity/v3/endpoints?service_id=f48e995cfe144c4c8dd2935a51ec8201&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:23.464849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919bd9c-95ec-409e-8aac-9cb64e765df9 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:23.503702 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:23.586623 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919bd9c-95ec-409e-8aac-9cb64e765df9 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:23.621918 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736f766-96af-4cc5-b9eb-c01f75852755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:23.623012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736f766-96af-4cc5-b9eb-c01f75852755 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:23.623495 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736f766-96af-4cc5-b9eb-c01f75852755 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:23.624005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736f766-96af-4cc5-b9eb-c01f75852755 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:23.624877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736f766-96af-4cc5-b9eb-c01f75852755 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:23.626434 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736f766-96af-4cc5-b9eb-c01f75852755 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:23.638977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 386/780] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 14 20:21:23 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:24.019675 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95d74d-1601-436e-a5a9-1a26af7e530b None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:24.020206 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95d74d-1601-436e-a5a9-1a26af7e530b None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:24.020206 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95d74d-1601-436e-a5a9-1a26af7e530b None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:24.021862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 395/781] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:24.026982 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d2f2d-4b06-4f04-9a03-414929af7cab None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:24.027450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d2f2d-4b06-4f04-9a03-414929af7cab None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:24.027558 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d2f2d-4b06-4f04-9a03-414929af7cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:24.047974 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-fb6d2f2d-4b06-4f04-9a03-414929af7cab None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:24.049182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fb6d2f2d-4b06-4f04-9a03-414929af7cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:24.137396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 387/782] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:24.174824 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95d74d-1601-436e-a5a9-1a26af7e530b None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:24.315371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95d74d-1601-436e-a5a9-1a26af7e530b None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:24.368612 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82883d7c-3247-48e0-8448-4ebc53e56b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:24.369413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82883d7c-3247-48e0-8448-4ebc53e56b7f None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:24.369696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82883d7c-3247-48e0-8448-4ebc53e56b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:24.369964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82883d7c-3247-48e0-8448-4ebc53e56b7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:24.392581 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82883d7c-3247-48e0-8448-4ebc53e56b7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:24.396813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-82883d7c-3247-48e0-8448-4ebc53e56b7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:21:24.396813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:24.396813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:24.396813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:24.396813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:24.396813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:24.396813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:24.396813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:24.396813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:24.396813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:24.396813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:24.396813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:24.396813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:24.396813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:24.396813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:24.396813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:24.397731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:24.397731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:24.397731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:24.397731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:24.397731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:24.397731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:24.397731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:24.397731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:24.397731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:24.397731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:24.397731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:24.397731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:24.397731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:24.397731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:24.397731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:24.397731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:24.398564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:24.398564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:24.398564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:24.398564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:24.398564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:24.398564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:24.398564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:24.398564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:24.398564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:24.398564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:24.398564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:21:24.398564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:24.401003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 396/783] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:21:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:24.412054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d2f2d-4b06-4f04-9a03-414929af7cab None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:24.443157 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:24.470238 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59aab3-06cc-4628-88ff-4cfa49aac7e1 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:24.470606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59aab3-06cc-4628-88ff-4cfa49aac7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:24.470848 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59aab3-06cc-4628-88ff-4cfa49aac7e1 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:24.472383 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 397/784] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:21:24.478203 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9c080-e531-4cb9-8508-c42a776fd898 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:24.478860 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9c080-e531-4cb9-8508-c42a776fd898 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:24.479387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9c080-e531-4cb9-8508-c42a776fd898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:24.498486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-89a9c080-e531-4cb9-8508-c42a776fd898 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:24.500192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-89a9c080-e531-4cb9-8508-c42a776fd898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:24.503480 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:24.579491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d2f2d-4b06-4f04-9a03-414929af7cab None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:24.602152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 398/785] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 20:21:24.639060 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746f37e-d086-4fc5-a995-beb920f13571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:24.639060 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746f37e-d086-4fc5-a995-beb920f13571 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:24.639060 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746f37e-d086-4fc5-a995-beb920f13571 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:24.639060 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746f37e-d086-4fc5-a995-beb920f13571 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:24.639447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746f37e-d086-4fc5-a995-beb920f13571 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:24.639447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746f37e-d086-4fc5-a995-beb920f13571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:24.639447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746f37e-d086-4fc5-a995-beb920f13571 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:24.648548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 388/786] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:21:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:24.653795 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9c080-e531-4cb9-8508-c42a776fd898 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:24.662248 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746f37e-d086-4fc5-a995-beb920f13571 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:24.775374 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:24.792130 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9c080-e531-4cb9-8508-c42a776fd898 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:24.807646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746f37e-d086-4fc5-a995-beb920f13571 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:24.831337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec462b-fa85-4217-9ed2-9276c1a1b72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:24.832628 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec462b-fa85-4217-9ed2-9276c1a1b72a None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:24.833056 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec462b-fa85-4217-9ed2-9276c1a1b72a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:24.833470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec462b-fa85-4217-9ed2-9276c1a1b72a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:24.844229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:24.847599 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec462b-fa85-4217-9ed2-9276c1a1b72a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:24.848429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7544648-7376-40e1-8cae-9c413dd0e141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:24.849462 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7544648-7376-40e1-8cae-9c413dd0e141 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:24.849849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7544648-7376-40e1-8cae-9c413dd0e141 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:24.850227 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7544648-7376-40e1-8cae-9c413dd0e141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:24.858004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-13ec462b-fa85-4217-9ed2-9276c1a1b72a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:24.858004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:24.858004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:24.858004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:24.858004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:24.858004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:24.858004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:24.858004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:24.858004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:24.858004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:24.858004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:24.858004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:24.858004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:24.858004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:24.858004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:24.858004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:24.859125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:24.859125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:24.859125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:24.859125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:24.859125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:24.859125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:24.859125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:24.859125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:24.859125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:24.859125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:24.859125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:24.859125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:24.859125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:24.859125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:24.859125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:24.859125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:24.860167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:24.860167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:24.860167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:24.860167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:24.860167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:24.860167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:24.860167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:24.860167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:24.860167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:24.860167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:24.860167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:24.860167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:24.863017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 399/787] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:21:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:24.864477 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef48d4f-9fa4-4127-baf4-905dc4fe1606 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:24.864902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef48d4f-9fa4-4127-baf4-905dc4fe1606 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:24.865318 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef48d4f-9fa4-4127-baf4-905dc4fe1606 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:24.871094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 400/788] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:24.872856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef48d4f-9fa4-4127-baf4-905dc4fe1606 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:24.874229 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7544648-7376-40e1-8cae-9c413dd0e141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:24.875721 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7544648-7376-40e1-8cae-9c413dd0e141 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:24.895901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-f7544648-7376-40e1-8cae-9c413dd0e141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:24.895901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:24.895901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:24.895901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:24.895901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:24.895901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:24.895901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:24.895901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:24.895901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:24.895901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:24.895901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:24.895901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:24.895901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:24.895901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:24.895901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:24.896832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:24.896832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:24.896832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:24.896832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:24.896832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:24.896832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:24.896832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:24.896832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:24.896832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:24.896832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:24.896832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:24.896832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:24.896832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:24.896832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:24.896832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:24.896832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:24.896832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:24.898352 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:24.898352 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:24.898352 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:24.898352 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:24.898352 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 389/789] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:24.898733 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9de97-fd56-492a-814a-782d2f815f12 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:24.899023 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9de97-fd56-492a-814a-782d2f815f12 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:24.899308 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9de97-fd56-492a-814a-782d2f815f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:24.918597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-3dc9de97-fd56-492a-814a-782d2f815f12 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:24.920219 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3dc9de97-fd56-492a-814a-782d2f815f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:24.998983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 390/790] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:24.998983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9de97-fd56-492a-814a-782d2f815f12 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:25.011422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef48d4f-9fa4-4127-baf4-905dc4fe1606 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:25.054038 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350cb5d0-0e4f-4c2d-9a1f-cc06546f01df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:25.055541 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cb5d0-0e4f-4c2d-9a1f-cc06546f01df None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:25.056029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cb5d0-0e4f-4c2d-9a1f-cc06546f01df None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:25.056564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cb5d0-0e4f-4c2d-9a1f-cc06546f01df None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:25.057571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350cb5d0-0e4f-4c2d-9a1f-cc06546f01df None admin] RBAC: Adding query filter params (name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:25.058035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350cb5d0-0e4f-4c2d-9a1f-cc06546f01df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:25.059669 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350cb5d0-0e4f-4c2d-9a1f-cc06546f01df None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:25.069768 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 401/791] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:21:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:25.076925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350cb5d0-0e4f-4c2d-9a1f-cc06546f01df None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:25.088088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:25.140673 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9de97-fd56-492a-814a-782d2f815f12 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:25.150880 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:25.185683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f986edd-06ee-405f-bae4-f1fc1ea6e882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:25.186503 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f986edd-06ee-405f-bae4-f1fc1ea6e882 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:25.186884 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f986edd-06ee-405f-bae4-f1fc1ea6e882 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:25.187294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f986edd-06ee-405f-bae4-f1fc1ea6e882 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:25.188232 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f986edd-06ee-405f-bae4-f1fc1ea6e882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:25.188888 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f986edd-06ee-405f-bae4-f1fc1ea6e882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:25.190540 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f986edd-06ee-405f-bae4-f1fc1ea6e882 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:25.198497 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350cb5d0-0e4f-4c2d-9a1f-cc06546f01df None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:25.205939 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 391/792] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:25.206857 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f986edd-06ee-405f-bae4-f1fc1ea6e882 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:25.244489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7a5ca-03c2-41b2-8c9a-92af673a55e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:25.245291 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7a5ca-03c2-41b2-8c9a-92af673a55e6 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:25.245654 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7a5ca-03c2-41b2-8c9a-92af673a55e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:25.245937 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7a5ca-03c2-41b2-8c9a-92af673a55e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:25.258357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7a5ca-03c2-41b2-8c9a-92af673a55e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:25.267425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-b9d7a5ca-03c2-41b2-8c9a-92af673a55e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:21:25.267425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:25.267425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:25.267425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:25.267425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:25.267425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:25.267425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:25.267425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:25.267425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:25.267425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:25.267425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:25.267425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:25.267425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:25.267425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:25.267425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:25.267425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:25.268401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:25.268401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:25.268401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:25.268401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:25.268401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:25.268401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:25.268401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:25.268401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:25.268401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:25.268401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:25.268401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:25.268401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:25.268401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:25.268401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:25.268401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:25.268401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:25.269505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:25.269505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:25.269505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:25.269505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:25.269505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:25.269505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:25.269505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:25.269505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:25.269505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:25.269505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:25.269505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:21:25.269505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:25.274000 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 402/793] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:21:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:25.274870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7a5ca-03c2-41b2-8c9a-92af673a55e6 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:25.328085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f986edd-06ee-405f-bae4-f1fc1ea6e882 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:25.352664 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:25.366227 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14d4e9-7e29-414b-b080-3f1362ecfb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:25.367473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14d4e9-7e29-414b-b080-3f1362ecfb5e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:25.367994 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14d4e9-7e29-414b-b080-3f1362ecfb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:25.368484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14d4e9-7e29-414b-b080-3f1362ecfb5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:25.380410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14d4e9-7e29-414b-b080-3f1362ecfb5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:25.382196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14d4e9-7e29-414b-b080-3f1362ecfb5e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:25.389122 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7a5ca-03c2-41b2-8c9a-92af673a55e6 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:25.401735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-ca14d4e9-7e29-414b-b080-3f1362ecfb5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:25.401735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:25.401735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:25.401735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:25.401735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:25.401735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:25.401735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:25.401735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:25.401735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:25.401735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:25.401735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:25.401735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:25.401735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:25.401735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:25.401735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:25.402646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:25.402646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:25.402646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:25.402646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:25.402646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:25.402646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:25.402646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:25.402646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:25.402646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:25.402646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:25.402646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:25.402646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:25.402646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:25.402646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:25.402646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:25.402646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:25.402646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:25.404196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:25.404196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:25.404196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:25.404196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:25.410758 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 392/794] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:25.411887 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14d4e9-7e29-414b-b080-3f1362ecfb5e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:25.413991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:25.427002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f1d88-419a-4e18-900d-5f28222f974f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:25.427819 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f1d88-419a-4e18-900d-5f28222f974f None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:25.428102 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f1d88-419a-4e18-900d-5f28222f974f None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:25.428478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f1d88-419a-4e18-900d-5f28222f974f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:25.437138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f1d88-419a-4e18-900d-5f28222f974f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:25.438819 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f1d88-419a-4e18-900d-5f28222f974f None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:25.455175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-313f1d88-419a-4e18-900d-5f28222f974f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 20:21:25.455175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:25.455175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:25.455175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:25.455175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:25.455175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:25.455175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:25.455175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:25.455175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:25.455175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:25.455175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:25.455175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:25.455175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:25.455175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:25.455175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:25.455175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:25.456186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:25.456186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:25.456186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:25.456186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:25.456186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:25.456186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:25.456186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:25.456186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:25.456186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:25.456186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:25.456186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:25.456186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:25.456186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:25.456186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:25.456186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:25.456186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:25.456186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:25.456186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:25.457162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:25.457162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:25.457162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:25.457162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:25.457162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:25.457162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:25.457162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:25.457162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:25.457162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 20:21:25.457162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:25.457162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 403/795] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 14 20:21:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:25.458257 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f1d88-419a-4e18-900d-5f28222f974f None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:25.526368 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14d4e9-7e29-414b-b080-3f1362ecfb5e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:25.564037 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ecb3a-c847-49df-91b1-da3f41ec9594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:25.565085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ecb3a-c847-49df-91b1-da3f41ec9594 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:25.565600 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ecb3a-c847-49df-91b1-da3f41ec9594 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:25.566084 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ecb3a-c847-49df-91b1-da3f41ec9594 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:25.567114 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ecb3a-c847-49df-91b1-da3f41ec9594 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:25.567630 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ecb3a-c847-49df-91b1-da3f41ec9594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:25.569309 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ecb3a-c847-49df-91b1-da3f41ec9594 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:25.570433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f1d88-419a-4e18-900d-5f28222f974f None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:25.580050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 393/796] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:21:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:25.581248 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ecb3a-c847-49df-91b1-da3f41ec9594 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:25.614955 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9cd0f-3152-4721-8554-6f9421d865bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:25.615672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9cd0f-3152-4721-8554-6f9421d865bb None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:25.615986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9cd0f-3152-4721-8554-6f9421d865bb None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:25.616311 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9cd0f-3152-4721-8554-6f9421d865bb None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:25.617047 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9cd0f-3152-4721-8554-6f9421d865bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:25.617352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9cd0f-3152-4721-8554-6f9421d865bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:25.618822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9cd0f-3152-4721-8554-6f9421d865bb None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:25.620579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:25.647092 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 404/797] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:25.648292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9cd0f-3152-4721-8554-6f9421d865bb None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:25.697967 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:25.699518 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ecb3a-c847-49df-91b1-da3f41ec9594 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:25.737229 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45549696-5260-43a4-a8f7-50eeb228a8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:25.737894 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45549696-5260-43a4-a8f7-50eeb228a8be None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:25.738178 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45549696-5260-43a4-a8f7-50eeb228a8be None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:25.738449 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45549696-5260-43a4-a8f7-50eeb228a8be None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:25.739362 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45549696-5260-43a4-a8f7-50eeb228a8be None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:25.739665 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45549696-5260-43a4-a8f7-50eeb228a8be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:25.741092 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45549696-5260-43a4-a8f7-50eeb228a8be None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:25.760794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 394/798] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 14 20:21:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:25.761712 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45549696-5260-43a4-a8f7-50eeb228a8be None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:25.768669 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9cd0f-3152-4721-8554-6f9421d865bb None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:25.811335 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bcfdb4-401f-451d-9e47-4f93f0e46f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:25.812137 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcfdb4-401f-451d-9e47-4f93f0e46f4b None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:25.812416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcfdb4-401f-451d-9e47-4f93f0e46f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:25.812971 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcfdb4-401f-451d-9e47-4f93f0e46f4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:25.823623 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bcfdb4-401f-451d-9e47-4f93f0e46f4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:25.824980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bcfdb4-401f-451d-9e47-4f93f0e46f4b None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:25.843351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-c8bcfdb4-401f-451d-9e47-4f93f0e46f4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:25.843351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:25.843351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:25.843351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:25.843351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:25.843351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:25.843351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:25.843351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:25.843351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:25.843351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:25.843351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:25.843351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:25.843351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:25.843351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:25.843351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:25.844330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:25.844330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:25.844330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:25.844330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:25.844330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:25.844330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:25.844330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:25.844330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:25.844330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:25.844330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:25.844330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:25.844330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:25.844330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:25.844330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:25.844330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:25.844330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:25.844330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:25.845322 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:25.845322 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:25.845322 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:25.845322 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:25.845322 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 405/799] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:25.846387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34493eae-3786-48f0-9c8b-b8a61d46811e None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:25.846387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34493eae-3786-48f0-9c8b-b8a61d46811e None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:25.846387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34493eae-3786-48f0-9c8b-b8a61d46811e None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:25.848005 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 406/800] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:25.848849 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34493eae-3786-48f0-9c8b-b8a61d46811e None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:25.866257 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:25.875306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45549696-5260-43a4-a8f7-50eeb228a8be None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:25.913569 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511b676-accb-4f6a-a237-d3dfd7b5a359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:25.914433 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511b676-accb-4f6a-a237-d3dfd7b5a359 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:25.914725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511b676-accb-4f6a-a237-d3dfd7b5a359 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:25.915032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511b676-accb-4f6a-a237-d3dfd7b5a359 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:25.923640 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511b676-accb-4f6a-a237-d3dfd7b5a359 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:25.925264 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511b676-accb-4f6a-a237-d3dfd7b5a359 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:25.942360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-f511b676-accb-4f6a-a237-d3dfd7b5a359 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 20:21:25.942360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:25.942360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:25.942360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:25.942360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:25.942360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:25.942360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:25.942360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:25.942360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:25.942360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:25.942360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:25.942360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:25.942360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:25.942360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:25.942360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:25.942360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:25.943396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:25.943396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:25.943396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:25.943396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:25.943396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:25.943396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:25.943396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:25.943396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:25.943396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:25.943396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:25.943396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:25.943396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:25.943396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:25.943396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:25.943396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:25.943396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:25.943396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:25.943396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:25.944335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:25.944335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:25.944335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:25.944335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:25.944335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:25.944335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:25.944335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:25.944335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:25.944335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 20:21:25.944335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:25.947285 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 395/801] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 14 20:21:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:25.948899 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684bbc0-f631-45a0-9709-2e58f52d5dad None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:25.949192 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684bbc0-f631-45a0-9709-2e58f52d5dad None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:25.949471 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684bbc0-f631-45a0-9709-2e58f52d5dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:25.956395 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:25.965368 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34493eae-3786-48f0-9c8b-b8a61d46811e None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:25.967325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-3684bbc0-f631-45a0-9709-2e58f52d5dad None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:25.968639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3684bbc0-f631-45a0-9709-2e58f52d5dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:26.005216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02f66a-f54f-4732-b500-b4a3a2a8d8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:26.005724 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02f66a-f54f-4732-b500-b4a3a2a8d8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:26.005957 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02f66a-f54f-4732-b500-b4a3a2a8d8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:26.006227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02f66a-f54f-4732-b500-b4a3a2a8d8c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:26.041776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 396/802] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:26.042628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684bbc0-f631-45a0-9709-2e58f52d5dad None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:26.079608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02f66a-f54f-4732-b500-b4a3a2a8d8c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:26.081045 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02f66a-f54f-4732-b500-b4a3a2a8d8c2 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:26.128538 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:26.144899 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684bbc0-f631-45a0-9709-2e58f52d5dad None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:26.173344 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:26.179773 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff7f44-9338-4889-a6bc-202571894382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:26.180443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff7f44-9338-4889-a6bc-202571894382 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:26.180721 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff7f44-9338-4889-a6bc-202571894382 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:26.180994 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff7f44-9338-4889-a6bc-202571894382 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:26.181313 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 407/803] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:21:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:21:26.181717 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff7f44-9338-4889-a6bc-202571894382 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:26.182019 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff7f44-9338-4889-a6bc-202571894382 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:26.182177 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02f66a-f54f-4732-b500-b4a3a2a8d8c2 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:26.183486 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff7f44-9338-4889-a6bc-202571894382 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:26.195929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 397/804] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:26.197256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90d915-f033-4e54-bf02-1e284b589aeb None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:26.197546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90d915-f033-4e54-bf02-1e284b589aeb None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:26.197818 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90d915-f033-4e54-bf02-1e284b589aeb None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:26.203097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 398/805] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:26.204036 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90d915-f033-4e54-bf02-1e284b589aeb None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:26.284147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02f66a-f54f-4732-b500-b4a3a2a8d8c2 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:26.312529 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90d915-f033-4e54-bf02-1e284b589aeb None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:26.319602 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b482fe-3db7-4720-821f-54a5ae6441e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:26.320332 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b482fe-3db7-4720-821f-54a5ae6441e2 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:26.320619 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b482fe-3db7-4720-821f-54a5ae6441e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:26.320894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b482fe-3db7-4720-821f-54a5ae6441e2 None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:26.321733 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b482fe-3db7-4720-821f-54a5ae6441e2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:26.322042 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b482fe-3db7-4720-821f-54a5ae6441e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:26.323519 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b482fe-3db7-4720-821f-54a5ae6441e2 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:26.342609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 408/806] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jul 14 20:21:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:26.343520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b482fe-3db7-4720-821f-54a5ae6441e2 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:26.355926 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387f59d-1e00-4145-a810-8cec86e340f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:26.357324 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387f59d-1e00-4145-a810-8cec86e340f0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:26.357614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387f59d-1e00-4145-a810-8cec86e340f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:26.357876 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387f59d-1e00-4145-a810-8cec86e340f0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:26.358539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387f59d-1e00-4145-a810-8cec86e340f0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:26.359960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387f59d-1e00-4145-a810-8cec86e340f0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:26.372332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-f387f59d-1e00-4145-a810-8cec86e340f0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 20:21:26.372332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:26.372332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:26.372332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:26.372332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:26.372332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:26.372332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:26.372332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:26.372332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 20:21:26.372332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:26.372332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:26.372332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:26.372332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:26.372332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:26.372332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:26.372332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:26.373422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:26.373422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:26.373422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:26.373422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:26.373422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:26.373422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:26.373422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:26.373422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:26.373422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:26.373422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:26.373422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:26.373422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:26.373422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:26.373422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:26.373422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:26.373422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:26.374252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:26.374252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:26.374252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:26.374252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:26.374252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:26.374252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:26.374252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:26.374252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:26.374252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:26.374252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:26.374252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:26.374252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:26.374252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:26.374252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 20:21:26.374252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:26.374252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 399/807] 10.222.0.52 () {56 vars in 1129 bytes} [Thu Jul 14 20:21:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:26.375163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387f59d-1e00-4145-a810-8cec86e340f0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:26.389534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:26.406185 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:26.454247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b482fe-3db7-4720-821f-54a5ae6441e2 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:26.482608 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387f59d-1e00-4145-a810-8cec86e340f0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:26.489886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbce37e-def2-4945-9ac3-850161d8ac58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:26.490685 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbce37e-def2-4945-9ac3-850161d8ac58 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:26.490991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbce37e-def2-4945-9ac3-850161d8ac58 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:26.491266 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbce37e-def2-4945-9ac3-850161d8ac58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:26.501575 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbce37e-def2-4945-9ac3-850161d8ac58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:26.503110 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbce37e-def2-4945-9ac3-850161d8ac58 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:26.517919 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899326a2-33e0-4c1a-9361-50d2247c163f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:26.518718 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899326a2-33e0-4c1a-9361-50d2247c163f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:26.519298 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899326a2-33e0-4c1a-9361-50d2247c163f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:26.519576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899326a2-33e0-4c1a-9361-50d2247c163f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:26.521737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-7bbce37e-def2-4945-9ac3-850161d8ac58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:26.521737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:26.521737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:26.521737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:26.521737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:26.521737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:26.521737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:26.521737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:26.521737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:26.521737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:26.521737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:26.521737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:26.521737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:26.521737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:26.521737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:26.522606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:26.522606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:26.522606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:26.522606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:26.522606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:26.522606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:26.522606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:26.522606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:26.522606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:26.522606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:26.522606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:26.522606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:26.522606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:26.522606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:26.522606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:26.522606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:26.522606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:26.523522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:26.523522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:26.523522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:26.523522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:26.523522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 409/808] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:26.524208 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dc274-af0e-4a2f-a04b-920aae042e21 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:26.524453 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dc274-af0e-4a2f-a04b-920aae042e21 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:26.524720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dc274-af0e-4a2f-a04b-920aae042e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:26.530847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899326a2-33e0-4c1a-9361-50d2247c163f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:26.532928 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899326a2-33e0-4c1a-9361-50d2247c163f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:26.543443 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-d60dc274-af0e-4a2f-a04b-920aae042e21 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:26.545254 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d60dc274-af0e-4a2f-a04b-920aae042e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:26.553239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-899326a2-33e0-4c1a-9361-50d2247c163f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 20:21:26.553239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:26.553239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:26.553239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:26.553239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:26.553239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:26.553239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:26.553239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:26.553239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:26.553239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:26.553239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:26.553239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:26.553239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:26.553239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:26.553239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:26.553239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:26.554745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:26.554745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:26.554745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:26.554745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:26.554745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:26.554745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:26.554745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:26.554745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:26.554745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:26.554745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:26.554745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:26.554745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:26.554745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:26.554745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:26.554745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:26.554745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:26.554745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:26.554745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:26.556545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:26.556545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:26.556545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:26.556545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:26.556545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:26.556545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:26.556545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:26.556545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:26.556545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 20:21:26.556545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:26.556545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 400/809] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:21:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:26.556545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899326a2-33e0-4c1a-9361-50d2247c163f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:26.620100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 410/810] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:26.620952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60dc274-af0e-4a2f-a04b-920aae042e21 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:26.637257 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:26.656963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:26.675238 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899326a2-33e0-4c1a-9361-50d2247c163f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:26.708907 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1164f-b33f-4c6f-887e-a3648579d084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:26.709702 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1164f-b33f-4c6f-887e-a3648579d084 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:26.709984 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1164f-b33f-4c6f-887e-a3648579d084 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:26.710250 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1164f-b33f-4c6f-887e-a3648579d084 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:26.722879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1164f-b33f-4c6f-887e-a3648579d084 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:26.724362 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1164f-b33f-4c6f-887e-a3648579d084 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:26.725132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60dc274-af0e-4a2f-a04b-920aae042e21 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:26.741730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-ddb1164f-b33f-4c6f-887e-a3648579d084 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:26.741730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:26.741730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:26.741730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:26.741730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:26.741730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:26.741730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:26.741730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:26.741730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:26.741730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:26.741730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:26.741730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:26.741730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:26.741730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:26.741730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:26.742739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:26.742739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:26.742739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:26.742739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:26.742739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:26.742739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:26.742739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:26.742739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:26.742739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:26.742739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:26.742739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:26.742739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:26.742739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:26.742739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:26.742739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:26.742739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:26.742739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:26.743676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:26.743676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:26.743676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:26.743676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:26.743676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 401/811] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:26.743676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1164f-b33f-4c6f-887e-a3648579d084 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:26.761735 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df0262-996d-41e2-ab91-6f3e2ab9c026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:26.762411 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df0262-996d-41e2-ab91-6f3e2ab9c026 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:26.763233 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df0262-996d-41e2-ab91-6f3e2ab9c026 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:26.763515 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df0262-996d-41e2-ab91-6f3e2ab9c026 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:26.764226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df0262-996d-41e2-ab91-6f3e2ab9c026 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:26.764536 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df0262-996d-41e2-ab91-6f3e2ab9c026 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:26.765904 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df0262-996d-41e2-ab91-6f3e2ab9c026 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:26.775150 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 411/812] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jul 14 20:21:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:26.775150 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df0262-996d-41e2-ab91-6f3e2ab9c026 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:26.845001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1164f-b33f-4c6f-887e-a3648579d084 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:26.866690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:26.875268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:26.879695 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df0262-996d-41e2-ab91-6f3e2ab9c026 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:26.881831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372f266-08df-495b-a0b2-85cec649523f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:26.882499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372f266-08df-495b-a0b2-85cec649523f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:26.882823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372f266-08df-495b-a0b2-85cec649523f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:26.883143 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372f266-08df-495b-a0b2-85cec649523f None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:26.883906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3372f266-08df-495b-a0b2-85cec649523f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:26.884247 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3372f266-08df-495b-a0b2-85cec649523f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:26.885680 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3372f266-08df-495b-a0b2-85cec649523f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:26.898999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 402/813] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:26.899757 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372f266-08df-495b-a0b2-85cec649523f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:26.922460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87275c66-8d6d-40f3-9d25-8787a37597f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:26.923166 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87275c66-8d6d-40f3-9d25-8787a37597f7 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:26.923456 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87275c66-8d6d-40f3-9d25-8787a37597f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:26.923734 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87275c66-8d6d-40f3-9d25-8787a37597f7 None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:26.924583 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87275c66-8d6d-40f3-9d25-8787a37597f7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:26.924894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87275c66-8d6d-40f3-9d25-8787a37597f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:26.926332 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87275c66-8d6d-40f3-9d25-8787a37597f7 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:26.944237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 412/814] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jul 14 20:21:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:26.944906 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87275c66-8d6d-40f3-9d25-8787a37597f7 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:27.001342 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372f266-08df-495b-a0b2-85cec649523f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:27.036608 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2518ff3-5c41-483c-94e6-0b80af2a9c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:27.037399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2518ff3-5c41-483c-94e6-0b80af2a9c91 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:27.037732 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2518ff3-5c41-483c-94e6-0b80af2a9c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:27.037956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2518ff3-5c41-483c-94e6-0b80af2a9c91 None admin] PATH_INFO: `/v3/services/network` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:27.038614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2518ff3-5c41-483c-94e6-0b80af2a9c91 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:27.040061 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2518ff3-5c41-483c-94e6-0b80af2a9c91 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:27.048441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87275c66-8d6d-40f3-9d25-8787a37597f7 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:27.052205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-c2518ff3-5c41-483c-94e6-0b80af2a9c91 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 20:21:27.052205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:27.052205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:27.052205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:27.052205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:27.052205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:27.052205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:27.052205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:27.052205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 20:21:27.052205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:27.052205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:27.052205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:27.052205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:27.052205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:27.052205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:27.052205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:27.052205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:27.053262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:27.053262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:27.053262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:27.053262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:27.053262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:27.053262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:27.053262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:27.053262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:27.053262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:27.053262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:27.053262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:27.053262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:27.053262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:27.053262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:27.053262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:27.053262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:27.054096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:27.054096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:27.054096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:27.054096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:27.054096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:27.054096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:27.054096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:27.054096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:27.054096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:27.054096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:27.054096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:27.054096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:27.054096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 20:21:27.054096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:27.054096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 403/815] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:26 2022] GET /identity/v3/services/network => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:27.055608 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2518ff3-5c41-483c-94e6-0b80af2a9c91 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:27.084139 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288edd57-01b7-4fa1-8757-ee84e75e054a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:27.084809 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288edd57-01b7-4fa1-8757-ee84e75e054a None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:27.085090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288edd57-01b7-4fa1-8757-ee84e75e054a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:27.085365 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288edd57-01b7-4fa1-8757-ee84e75e054a None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:27.086091 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288edd57-01b7-4fa1-8757-ee84e75e054a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:27.086505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288edd57-01b7-4fa1-8757-ee84e75e054a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:27.087799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288edd57-01b7-4fa1-8757-ee84e75e054a None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:27.099251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 413/816] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:27.100078 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288edd57-01b7-4fa1-8757-ee84e75e054a None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:27.111124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:27.118991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:27.168329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2518ff3-5c41-483c-94e6-0b80af2a9c91 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:27.202622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b2081-ba1d-4928-8c64-dc2867294e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:27.203327 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b2081-ba1d-4928-8c64-dc2867294e0a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:27.203616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b2081-ba1d-4928-8c64-dc2867294e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:27.203896 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b2081-ba1d-4928-8c64-dc2867294e0a None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:27.204608 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b2081-ba1d-4928-8c64-dc2867294e0a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:27.204914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b2081-ba1d-4928-8c64-dc2867294e0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:27.205030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288edd57-01b7-4fa1-8757-ee84e75e054a None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:27.206298 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b2081-ba1d-4928-8c64-dc2867294e0a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:27.211373 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 404/817] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jul 14 20:21:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:27.212168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b2081-ba1d-4928-8c64-dc2867294e0a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:27.240242 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee353a-5bbf-4ad7-bf26-c9809bf7e5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:27.241040 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee353a-5bbf-4ad7-bf26-c9809bf7e5da None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:27.241325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee353a-5bbf-4ad7-bf26-c9809bf7e5da None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:27.241608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee353a-5bbf-4ad7-bf26-c9809bf7e5da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:27.248900 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee353a-5bbf-4ad7-bf26-c9809bf7e5da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:27.250372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee353a-5bbf-4ad7-bf26-c9809bf7e5da None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:27.265859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-39ee353a-5bbf-4ad7-bf26-c9809bf7e5da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:27.265859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:27.265859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:27.265859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:27.265859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:27.265859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:27.265859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:27.265859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:27.265859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:27.265859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:27.265859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:27.265859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:27.265859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:27.265859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:27.265859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:27.267029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:27.267029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:27.267029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:27.267029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:27.267029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:27.267029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:27.267029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:27.267029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:27.267029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:27.267029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:27.267029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:27.267029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:27.267029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:27.267029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:27.267029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:27.267029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:27.267029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:27.267938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:27.267938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:27.267938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:27.267938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:27.267938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:27.267938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:27.267938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:27.267938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:27.267938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 414/818] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:27.267938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee353a-5bbf-4ad7-bf26-c9809bf7e5da None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:27.320478 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b2081-ba1d-4928-8c64-dc2867294e0a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:27.330756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:27.348901 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:27.358884 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ed784-4241-4ca5-9809-b3a8b2b0e86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:27.359703 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ed784-4241-4ca5-9809-b3a8b2b0e86b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:27.359984 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ed784-4241-4ca5-9809-b3a8b2b0e86b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:27.360261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ed784-4241-4ca5-9809-b3a8b2b0e86b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:27.376435 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ed784-4241-4ca5-9809-b3a8b2b0e86b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:27.377620 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee353a-5bbf-4ad7-bf26-c9809bf7e5da None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:27.378955 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ed784-4241-4ca5-9809-b3a8b2b0e86b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:27.408576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-280ed784-4241-4ca5-9809-b3a8b2b0e86b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:27.408576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:27.408576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:27.408576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:27.408576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:27.408576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:27.408576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:27.408576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:27.408576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:27.408576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:27.408576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:27.408576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:27.408576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:27.408576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:27.408576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:27.409804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:27.409804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:27.409804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:27.409804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:27.409804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:27.409804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:27.409804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:27.409804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:27.409804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:27.409804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:27.409804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:27.409804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:27.409804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:27.409804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:27.409804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:27.409804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:27.409804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:27.410794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:27.410794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:27.410794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:27.410794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:27.410794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 405/819] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:27.412123 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ed784-4241-4ca5-9809-b3a8b2b0e86b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:27.427225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f06625-6b8a-41d2-9251-fab84277fedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:27.427823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f06625-6b8a-41d2-9251-fab84277fedb None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:27.428102 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f06625-6b8a-41d2-9251-fab84277fedb None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:27.428378 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f06625-6b8a-41d2-9251-fab84277fedb None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:27.429084 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f06625-6b8a-41d2-9251-fab84277fedb None admin] RBAC: Adding query filter params (name=network) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:27.429386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f06625-6b8a-41d2-9251-fab84277fedb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:27.430760 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f06625-6b8a-41d2-9251-fab84277fedb None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:27.436310 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 415/820] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 14 20:21:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:27.436860 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f06625-6b8a-41d2-9251-fab84277fedb None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:27.520289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ed784-4241-4ca5-9809-b3a8b2b0e86b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:27.533746 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f06625-6b8a-41d2-9251-fab84277fedb None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:27.555429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2456af5-0dad-4c7f-bbeb-6812ae35c2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:27.556236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2456af5-0dad-4c7f-bbeb-6812ae35c2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:27.556518 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2456af5-0dad-4c7f-bbeb-6812ae35c2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:27.556788 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2456af5-0dad-4c7f-bbeb-6812ae35c2c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:27.564152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2456af5-0dad-4c7f-bbeb-6812ae35c2c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:27.565588 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2456af5-0dad-4c7f-bbeb-6812ae35c2c4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:27.569149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3710c-da5a-41e4-b42a-60663f36ed4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:27.569810 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3710c-da5a-41e4-b42a-60663f36ed4e None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:27.570094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3710c-da5a-41e4-b42a-60663f36ed4e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:27.570371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3710c-da5a-41e4-b42a-60663f36ed4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:27.571212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3710c-da5a-41e4-b42a-60663f36ed4e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:27.572561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3710c-da5a-41e4-b42a-60663f36ed4e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:27.579313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-b2456af5-0dad-4c7f-bbeb-6812ae35c2c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:27.579313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:27.579313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:27.579313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:27.579313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:27.579313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:27.579313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:27.579313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:27.579313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:27.579313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:27.579313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:27.579313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:27.579313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:27.579313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:27.579313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:27.580323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:27.580323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:27.580323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:27.580323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:27.580323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:27.580323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:27.580323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:27.580323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:27.580323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:27.580323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:27.580323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:27.580323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:27.580323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:27.580323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:27.580323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:27.580323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:27.580323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:27.581294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:27.581294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:27.581294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:27.581294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:27.581294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:27.581294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:27.581294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:27.581294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:27.581294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 406/821] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:27.581776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2456af5-0dad-4c7f-bbeb-6812ae35c2c4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:27.582459 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:27.598446 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 416/822] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 14 20:21:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:21:27.599299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3710c-da5a-41e4-b42a-60663f36ed4e None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:27.627870 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:27.735953 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3710c-da5a-41e4-b42a-60663f36ed4e None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:27.742257 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2456af5-0dad-4c7f-bbeb-6812ae35c2c4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:27.780139 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e403ce1-1e5e-4fe0-93f7-c48fe53b1d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:27.780827 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e403ce1-1e5e-4fe0-93f7-c48fe53b1d05 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:27.781107 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e403ce1-1e5e-4fe0-93f7-c48fe53b1d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:27.781383 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e403ce1-1e5e-4fe0-93f7-c48fe53b1d05 None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:27.782121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e403ce1-1e5e-4fe0-93f7-c48fe53b1d05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:27.782426 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e403ce1-1e5e-4fe0-93f7-c48fe53b1d05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:27.783836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e403ce1-1e5e-4fe0-93f7-c48fe53b1d05 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:27.787670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5649f-d723-4567-b42a-f38ab473538f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:27.788378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5649f-d723-4567-b42a-f38ab473538f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:27.788652 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5649f-d723-4567-b42a-f38ab473538f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:27.788922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5649f-d723-4567-b42a-f38ab473538f None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:27.789745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5649f-d723-4567-b42a-f38ab473538f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:27.790048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5649f-d723-4567-b42a-f38ab473538f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:27.791545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5649f-d723-4567-b42a-f38ab473538f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:27.803425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 417/823] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:27.804290 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e403ce1-1e5e-4fe0-93f7-c48fe53b1d05 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:27.809296 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 407/824] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:27.811180 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5649f-d723-4567-b42a-f38ab473538f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:27.863732 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:27.902311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:27.919461 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e403ce1-1e5e-4fe0-93f7-c48fe53b1d05 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:27.925128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5649f-d723-4567-b42a-f38ab473538f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:27.955022 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cc530-ddd3-4165-bef5-34483ead3f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:27.955695 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cc530-ddd3-4165-bef5-34483ead3f94 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:27.955980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cc530-ddd3-4165-bef5-34483ead3f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:27.956257 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cc530-ddd3-4165-bef5-34483ead3f94 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:27.956973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cc530-ddd3-4165-bef5-34483ead3f94 None admin] RBAC: Adding query filter params (type=network) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:27.957285 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cc530-ddd3-4165-bef5-34483ead3f94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:27.958665 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cc530-ddd3-4165-bef5-34483ead3f94 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:27.963579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e015ae-8e68-472d-aa0a-135e128d3e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:27.963724 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 418/825] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 14 20:21:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:27.964488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cc530-ddd3-4165-bef5-34483ead3f94 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:27.964742 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e015ae-8e68-472d-aa0a-135e128d3e93 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:27.965246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e015ae-8e68-472d-aa0a-135e128d3e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:27.966222 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e015ae-8e68-472d-aa0a-135e128d3e93 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:27.967222 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e015ae-8e68-472d-aa0a-135e128d3e93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:27.967703 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e015ae-8e68-472d-aa0a-135e128d3e93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:27.969206 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e015ae-8e68-472d-aa0a-135e128d3e93 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:27.987051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 408/826] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:27.987051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e015ae-8e68-472d-aa0a-135e128d3e93 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:28.086638 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cc530-ddd3-4165-bef5-34483ead3f94 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:28.096846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e015ae-8e68-472d-aa0a-135e128d3e93 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:28.105220 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:28.126407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d6798-7bb2-4ee2-9dd8-a8cba1071ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:28.127359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d6798-7bb2-4ee2-9dd8-a8cba1071ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:28.127642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d6798-7bb2-4ee2-9dd8-a8cba1071ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:28.127920 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d6798-7bb2-4ee2-9dd8-a8cba1071ef6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:28.133731 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24bec6-5069-4c75-bee6-0129c764329e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:28.134398 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24bec6-5069-4c75-bee6-0129c764329e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:28.134682 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24bec6-5069-4c75-bee6-0129c764329e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:28.134980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24bec6-5069-4c75-bee6-0129c764329e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:28.135439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d6798-7bb2-4ee2-9dd8-a8cba1071ef6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:28.135833 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24bec6-5069-4c75-bee6-0129c764329e None admin] RBAC: Adding query filter params (role.id=5ca112885467422684e134cb0f94865a, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=aa3bb9f1e24a4e9ebd460e97b3736bc0) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:28.136145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24bec6-5069-4c75-bee6-0129c764329e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:28.136906 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d6798-7bb2-4ee2-9dd8-a8cba1071ef6 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:28.137618 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24bec6-5069-4c75-bee6-0129c764329e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:28.144422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 409/827] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:21:27 2022] GET /identity/v3/role_assignments?user.id=aa3bb9f1e24a4e9ebd460e97b3736bc0&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=5ca112885467422684e134cb0f94865a => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:28.145249 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24bec6-5069-4c75-bee6-0129c764329e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:28.160020 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:28.164359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-fb0d6798-7bb2-4ee2-9dd8-a8cba1071ef6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:28.164359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:28.164359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:28.164359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:28.164359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:28.164359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:28.164359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:28.164359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:28.164359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:28.164359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:28.164359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:28.164359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:28.164359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:28.164359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:28.164359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:28.165473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:28.165473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:28.165473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:28.165473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:28.165473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:28.165473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:28.165473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:28.165473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:28.165473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:28.165473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:28.165473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:28.165473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:28.165473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:28.165473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:28.165473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:28.165473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:28.165473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:28.167221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:28.167221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:28.167221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:28.167221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:28.167221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:28.167221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:28.167221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:28.167221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:28.171475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 419/828] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:28.172255 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d6798-7bb2-4ee2-9dd8-a8cba1071ef6 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:28.287517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24bec6-5069-4c75-bee6-0129c764329e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:28.313149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d6798-7bb2-4ee2-9dd8-a8cba1071ef6 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:28.331922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c359ff6-396e-4f79-94ae-4e52ebc82917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:28.333002 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c359ff6-396e-4f79-94ae-4e52ebc82917 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:28.333446 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c359ff6-396e-4f79-94ae-4e52ebc82917 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:28.333870 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c359ff6-396e-4f79-94ae-4e52ebc82917 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:28.334992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c359ff6-396e-4f79-94ae-4e52ebc82917 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:28.336477 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c359ff6-396e-4f79-94ae-4e52ebc82917 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:28.361148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824f990-de50-4326-906e-00274e8a2ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:28.362126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 410/829] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 14 20:21:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:21:28.362238 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824f990-de50-4326-906e-00274e8a2ffd None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:28.362438 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824f990-de50-4326-906e-00274e8a2ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:28.362748 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824f990-de50-4326-906e-00274e8a2ffd None admin] PATH_INFO: `/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/4f75b8b484384554ad59aa20535875e9/roles/ea49c36b4a7f4cfb9048065638d458af` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:28.362921 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c359ff6-396e-4f79-94ae-4e52ebc82917 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:28.391100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824f990-de50-4326-906e-00274e8a2ffd None admin] RBAC: Authorizing `identity:create_grant(project_id=f33c8ac5154f46a89d1a0dc11fb58cf4, user_id=4f75b8b484384554ad59aa20535875e9, role_id=ea49c36b4a7f4cfb9048065638d458af)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:28.393393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824f990-de50-4326-906e-00274e8a2ffd None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:28.417345 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 420/830] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:21:28 2022] PUT /identity/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/4f75b8b484384554ad59aa20535875e9/roles/ea49c36b4a7f4cfb9048065638d458af => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:21:28.447058 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:28.514299 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c359ff6-396e-4f79-94ae-4e52ebc82917 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:28.556988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef22e7-5d05-48fd-9ac5-5ebb39557a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:28.557606 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef22e7-5d05-48fd-9ac5-5ebb39557a31 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:28.557893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef22e7-5d05-48fd-9ac5-5ebb39557a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:28.558173 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef22e7-5d05-48fd-9ac5-5ebb39557a31 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:28.559069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef22e7-5d05-48fd-9ac5-5ebb39557a31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:28.559388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef22e7-5d05-48fd-9ac5-5ebb39557a31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:28.561456 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef22e7-5d05-48fd-9ac5-5ebb39557a31 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:28.583074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 411/831] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:28.585053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824f990-de50-4326-906e-00274e8a2ffd None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:28.626586 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:28.723785 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824f990-de50-4326-906e-00274e8a2ffd None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:28.769174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6c9a4-cb39-452a-9a83-15e4062dbcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:28.770530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6c9a4-cb39-452a-9a83-15e4062dbcdd None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:28.771047 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6c9a4-cb39-452a-9a83-15e4062dbcdd None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:28.771528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6c9a4-cb39-452a-9a83-15e4062dbcdd None admin] PATH_INFO: `/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/acc23233929144a5b7b3a1733da19828/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:28.775756 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd6c9a4-cb39-452a-9a83-15e4062dbcdd None admin] RBAC: Authorizing `identity:create_grant(project_id=f33c8ac5154f46a89d1a0dc11fb58cf4, user_id=acc23233929144a5b7b3a1733da19828, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:28.778310 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd6c9a4-cb39-452a-9a83-15e4062dbcdd None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:28.802631 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 421/832] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:21:28 2022] PUT /identity/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/acc23233929144a5b7b3a1733da19828/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:21:29.730919 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73e4dd-ed81-4d1e-a089-10ca122fd383 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:29.732023 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73e4dd-ed81-4d1e-a089-10ca122fd383 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:29.732023 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73e4dd-ed81-4d1e-a089-10ca122fd383 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:29.733750 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 412/833] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:29.739425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce82360-0c37-4a1d-8111-28f065c5770e None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:29.739941 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce82360-0c37-4a1d-8111-28f065c5770e None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:29.740077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce82360-0c37-4a1d-8111-28f065c5770e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:29.757983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-7ce82360-0c37-4a1d-8111-28f065c5770e None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:29.759235 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-7ce82360-0c37-4a1d-8111-28f065c5770e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:29.837058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 422/834] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:29.858908 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73e4dd-ed81-4d1e-a089-10ca122fd383 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:29.971392 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73e4dd-ed81-4d1e-a089-10ca122fd383 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:29.991458 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:30.011579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b12153-f1b9-4964-bbd0-4f1fb46f6295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:30.012392 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b12153-f1b9-4964-bbd0-4f1fb46f6295 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:30.013212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b12153-f1b9-4964-bbd0-4f1fb46f6295 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:30.013494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b12153-f1b9-4964-bbd0-4f1fb46f6295 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:30.014162 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b12153-f1b9-4964-bbd0-4f1fb46f6295 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:30.015579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b12153-f1b9-4964-bbd0-4f1fb46f6295 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:30.029091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-62b12153-f1b9-4964-bbd0-4f1fb46f6295 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 20:21:30.029091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:30.029091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:30.029091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:30.029091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:30.029091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:30.029091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:30.029091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:30.029091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 20:21:30.029091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:30.029091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:30.029091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:30.029091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:30.029091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:30.029091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:30.029091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:30.029091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:30.031354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:30.031354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:30.031354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:30.031354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:30.031354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:30.031354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:30.031354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:30.031354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:30.031354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:30.031354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:30.031354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:30.031354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:30.031354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:30.031354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:30.031354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:30.031354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:30.033404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:30.033404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:30.033404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:30.033404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:30.033404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:30.033404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:30.033404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:30.033404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:30.033404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:30.033404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:30.033404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:30.033404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:30.033404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 20:21:30.033404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:30.039442 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 413/835] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:21:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:30.040300 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce82360-0c37-4a1d-8111-28f065c5770e None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:30.058746 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:30.151369 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce82360-0c37-4a1d-8111-28f065c5770e None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:30.188312 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f47bae-f29c-43e5-8108-d72524d72a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:30.188992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f47bae-f29c-43e5-8108-d72524d72a05 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:30.189277 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f47bae-f29c-43e5-8108-d72524d72a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:30.189555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f47bae-f29c-43e5-8108-d72524d72a05 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:30.190277 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f47bae-f29c-43e5-8108-d72524d72a05 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:30.190584 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f47bae-f29c-43e5-8108-d72524d72a05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:30.192046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f47bae-f29c-43e5-8108-d72524d72a05 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:30.197663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 423/836] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:21:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:30.204005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b12153-f1b9-4964-bbd0-4f1fb46f6295 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:30.312654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b12153-f1b9-4964-bbd0-4f1fb46f6295 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:30.353296 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e51974-8e12-4f41-99be-12ec5f2b61a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:30.353962 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e51974-8e12-4f41-99be-12ec5f2b61a7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:30.354243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e51974-8e12-4f41-99be-12ec5f2b61a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:30.354514 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e51974-8e12-4f41-99be-12ec5f2b61a7 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:30.355264 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e51974-8e12-4f41-99be-12ec5f2b61a7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:30.355569 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e51974-8e12-4f41-99be-12ec5f2b61a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:30.357037 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e51974-8e12-4f41-99be-12ec5f2b61a7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:30.362616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 414/837] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:21:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:30.623469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b0613-b117-46c4-b483-a74bb0b29a1e None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:30.623686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b0613-b117-46c4-b483-a74bb0b29a1e None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:30.623975 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b0613-b117-46c4-b483-a74bb0b29a1e None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:30.625637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 424/838] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:30 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:30.635451 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07661989-1189-407b-83f3-cf35f8509d63 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:30.635451 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07661989-1189-407b-83f3-cf35f8509d63 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:30.635451 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07661989-1189-407b-83f3-cf35f8509d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:30.646241 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df20921-848b-4b0e-8444-049e2a87f68e None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:30.646504 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df20921-848b-4b0e-8444-049e2a87f68e None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:30.646857 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df20921-848b-4b0e-8444-049e2a87f68e None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:30.648501 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 425/839] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:30.654248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5526c9-f481-4548-9940-93a815040152 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:30.654542 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5526c9-f481-4548-9940-93a815040152 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:30.654815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5526c9-f481-4548-9940-93a815040152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:30.660439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-07661989-1189-407b-83f3-cf35f8509d63 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:30.662238 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-07661989-1189-407b-83f3-cf35f8509d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:30.673358 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-8f5526c9-f481-4548-9940-93a815040152 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:30.674591 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-8f5526c9-f481-4548-9940-93a815040152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:30.717422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:30.753862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 415/840] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:30.756113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 426/841] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:30.778755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07661989-1189-407b-83f3-cf35f8509d63 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:30.789635 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5526c9-f481-4548-9940-93a815040152 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:30.807939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:30.913683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07661989-1189-407b-83f3-cf35f8509d63 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:30.936225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5526c9-f481-4548-9940-93a815040152 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:30.957087 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6b7e1-ffe9-4058-845c-ab49ff74eb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:30.957914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6b7e1-ffe9-4058-845c-ab49ff74eb67 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:30.958198 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6b7e1-ffe9-4058-845c-ab49ff74eb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:30.958477 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6b7e1-ffe9-4058-845c-ab49ff74eb67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:30.971995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6b7e1-ffe9-4058-845c-ab49ff74eb67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:30.984259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-cde6b7e1-ffe9-4058-845c-ab49ff74eb67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:30.984259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:30.984259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:30.984259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:30.984259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:30.984259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:30.984259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:30.984259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:30.984259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:30.984259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:30.984259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:30.984259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:30.984259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:30.984259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:30.984259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:30.984259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:30.985300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:30.985300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:30.985300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:30.985300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:30.985300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:30.985300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:30.985300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:30.985300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:30.985300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:30.985300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:30.985300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:30.985300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:30.985300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:30.985300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:30.985300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:30.985300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:30.986142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:30.986142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:30.986142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:30.986142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:30.986142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:30.986142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:30.986142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:30.986142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:30.986142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:30.986142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:30.986142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:21:30.986142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:30.986767 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 416/842] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:21:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:30.990229 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3d9f0-59ce-4a37-b2a0-b6a6fc720e66 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:30.990508 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3d9f0-59ce-4a37-b2a0-b6a6fc720e66 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:30.990782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3d9f0-59ce-4a37-b2a0-b6a6fc720e66 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:30.992325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 417/843] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:30.993124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3d9f0-59ce-4a37-b2a0-b6a6fc720e66 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:31.036355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:31.099447 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d83b8-b052-4331-9e1e-fdaf1bd95dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:31.100555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d83b8-b052-4331-9e1e-fdaf1bd95dfe None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:31.100980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d83b8-b052-4331-9e1e-fdaf1bd95dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:31.101396 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d83b8-b052-4331-9e1e-fdaf1bd95dfe None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:31.102207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d83b8-b052-4331-9e1e-fdaf1bd95dfe None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:31.103595 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d83b8-b052-4331-9e1e-fdaf1bd95dfe None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:31.112728 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3d9f0-59ce-4a37-b2a0-b6a6fc720e66 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:31.116331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-f73d83b8-b052-4331-9e1e-fdaf1bd95dfe None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 20:21:31.116331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:31.116331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:31.116331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:31.116331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:31.116331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:31.116331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:31.116331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:31.116331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 20:21:31.116331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:31.116331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:31.116331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:31.116331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:31.116331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:31.116331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:31.116331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:31.117279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:31.117279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:31.117279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:31.117279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:31.117279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:31.117279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:31.117279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:31.117279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:31.117279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:31.117279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:31.117279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:31.117279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:31.117279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:31.117279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:31.117279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:31.117279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:31.123460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:31.123460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:31.123460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:31.123460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:31.123460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:31.123460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:31.123460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:31.123460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:31.123460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:31.123460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:31.123460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:31.123460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:31.123460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:31.123460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 20:21:31.123460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:31.123460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 427/844] 10.222.0.52 () {56 vars in 1132 bytes} [Thu Jul 14 20:21:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:21:31.124362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce1bc6-fc80-48b3-b672-774253806aa4 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:31.124362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce1bc6-fc80-48b3-b672-774253806aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:31.124362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce1bc6-fc80-48b3-b672-774253806aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:31.137029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-afce1bc6-fc80-48b3-b672-774253806aa4 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:31.138267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-afce1bc6-fc80-48b3-b672-774253806aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:31.159054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877a28e-b744-4e04-8224-a1467fe23205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:31.159726 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877a28e-b744-4e04-8224-a1467fe23205 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:31.160008 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877a28e-b744-4e04-8224-a1467fe23205 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:31.160286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877a28e-b744-4e04-8224-a1467fe23205 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:31.161039 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8877a28e-b744-4e04-8224-a1467fe23205 None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:31.161340 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8877a28e-b744-4e04-8224-a1467fe23205 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:31.162702 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8877a28e-b744-4e04-8224-a1467fe23205 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:31.171849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 418/845] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:21:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:31.172578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877a28e-b744-4e04-8224-a1467fe23205 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:31.220747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 428/846] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:31.221927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce1bc6-fc80-48b3-b672-774253806aa4 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:31.228681 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:31.293654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:31.300876 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877a28e-b744-4e04-8224-a1467fe23205 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:31.338038 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d24c2-351e-4cfe-a0fc-ad1ef6fefdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:31.339120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d24c2-351e-4cfe-a0fc-ad1ef6fefdba None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:31.339579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d24c2-351e-4cfe-a0fc-ad1ef6fefdba None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:31.340025 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d24c2-351e-4cfe-a0fc-ad1ef6fefdba None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:31.341472 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d24c2-351e-4cfe-a0fc-ad1ef6fefdba None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:31.341760 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce1bc6-fc80-48b3-b672-774253806aa4 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:31.342193 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d24c2-351e-4cfe-a0fc-ad1ef6fefdba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:31.343908 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d24c2-351e-4cfe-a0fc-ad1ef6fefdba None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:31.349448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 419/847] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 14 20:21:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:31.350521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d24c2-351e-4cfe-a0fc-ad1ef6fefdba None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:31.382440 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eafc32-0e14-4f44-99d4-7536fed1b9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:31.383376 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eafc32-0e14-4f44-99d4-7536fed1b9ba None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:31.383548 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eafc32-0e14-4f44-99d4-7536fed1b9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:31.383823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eafc32-0e14-4f44-99d4-7536fed1b9ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:31.395210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eafc32-0e14-4f44-99d4-7536fed1b9ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:31.396655 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eafc32-0e14-4f44-99d4-7536fed1b9ba None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:31.414796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-b7eafc32-0e14-4f44-99d4-7536fed1b9ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:31.414796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:31.414796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:31.414796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:31.414796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:31.414796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:31.414796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:31.414796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:31.414796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:31.414796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:31.414796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:31.414796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:31.414796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:31.414796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:31.414796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:31.415943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:31.415943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:31.415943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:31.415943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:31.415943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:31.415943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:31.415943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:31.415943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:31.415943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:31.415943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:31.415943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:31.415943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:31.415943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:31.415943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:31.415943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:31.415943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:31.415943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:31.416867 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:31.416867 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:31.416867 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:31.416867 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:31.416867 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 429/848] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:31.416867 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eafc32-0e14-4f44-99d4-7536fed1b9ba None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:31.470501 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d24c2-351e-4cfe-a0fc-ad1ef6fefdba None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:31.480378 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:31.509409 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03283eb-1504-49f6-ae16-98e0e9f7147e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:31.510247 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03283eb-1504-49f6-ae16-98e0e9f7147e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:31.510536 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03283eb-1504-49f6-ae16-98e0e9f7147e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:31.510816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03283eb-1504-49f6-ae16-98e0e9f7147e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:31.524077 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03283eb-1504-49f6-ae16-98e0e9f7147e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:31.532338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-d03283eb-1504-49f6-ae16-98e0e9f7147e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:21:31.532338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:31.532338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:31.532338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:31.532338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:31.532338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:31.532338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:31.532338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:31.532338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:31.532338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:31.532338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:31.532338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:31.532338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:31.532338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:31.532338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:31.532338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:31.533288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:31.533288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:31.533288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:31.533288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:31.533288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:31.533288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:31.533288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:31.533288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:31.533288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:31.533288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:31.533288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:31.533288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:31.533288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:31.533288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:31.533288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:31.533288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:31.539647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:31.539647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:31.539647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:31.539647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:31.539647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:31.539647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:31.539647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:31.539647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:31.539647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:31.539647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:31.539647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:21:31.539647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:31.539647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 420/849] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:21:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:31.540515 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eafc32-0e14-4f44-99d4-7536fed1b9ba None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:31.540616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03283eb-1504-49f6-ae16-98e0e9f7147e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:31.563511 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:31.582346 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006bb6d-adb6-4add-a8fa-2e0b84f0cb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:31.583055 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006bb6d-adb6-4add-a8fa-2e0b84f0cb0a None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:31.583345 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006bb6d-adb6-4add-a8fa-2e0b84f0cb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:31.583624 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006bb6d-adb6-4add-a8fa-2e0b84f0cb0a None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:31.584721 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006bb6d-adb6-4add-a8fa-2e0b84f0cb0a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:31.584721 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006bb6d-adb6-4add-a8fa-2e0b84f0cb0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:31.586046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006bb6d-adb6-4add-a8fa-2e0b84f0cb0a None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:31.591199 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 430/850] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 14 20:21:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:31.592007 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006bb6d-adb6-4add-a8fa-2e0b84f0cb0a None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:31.711780 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03283eb-1504-49f6-ae16-98e0e9f7147e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:31.742540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006bb6d-adb6-4add-a8fa-2e0b84f0cb0a None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:31.754019 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06513d05-18f6-4ce8-8469-fa1f884f557a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:31.754739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06513d05-18f6-4ce8-8469-fa1f884f557a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:31.755016 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06513d05-18f6-4ce8-8469-fa1f884f557a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:31.755326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06513d05-18f6-4ce8-8469-fa1f884f557a None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:31.756082 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06513d05-18f6-4ce8-8469-fa1f884f557a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:31.756345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06513d05-18f6-4ce8-8469-fa1f884f557a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:31.757758 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06513d05-18f6-4ce8-8469-fa1f884f557a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:31.771948 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 421/851] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:31.774923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:31.778381 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06513d05-18f6-4ce8-8469-fa1f884f557a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:31.800507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2c4f7-64e1-4e6d-845c-3ba7026598b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:31.801184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2c4f7-64e1-4e6d-845c-3ba7026598b2 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:31.801470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2c4f7-64e1-4e6d-845c-3ba7026598b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:31.801746 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2c4f7-64e1-4e6d-845c-3ba7026598b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:31.802504 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff2c4f7-64e1-4e6d-845c-3ba7026598b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:31.802818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff2c4f7-64e1-4e6d-845c-3ba7026598b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:31.804249 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff2c4f7-64e1-4e6d-845c-3ba7026598b2 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:31.816402 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 431/852] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:21:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:31.822485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2c4f7-64e1-4e6d-845c-3ba7026598b2 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:31.890003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:31.916475 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06513d05-18f6-4ce8-8469-fa1f884f557a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:31.949587 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2c4f7-64e1-4e6d-845c-3ba7026598b2 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:31.964427 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62abeb-0b84-4939-9dde-83b760c43508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:31.966011 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62abeb-0b84-4939-9dde-83b760c43508 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:31.966757 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62abeb-0b84-4939-9dde-83b760c43508 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:31.967253 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62abeb-0b84-4939-9dde-83b760c43508 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:31.978426 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62abeb-0b84-4939-9dde-83b760c43508 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:31.981854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62abeb-0b84-4939-9dde-83b760c43508 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:31.992101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6c015-90e8-4094-b672-8c344f5125cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:31.993294 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6c015-90e8-4094-b672-8c344f5125cb None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:31.993623 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6c015-90e8-4094-b672-8c344f5125cb None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:31.993971 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6c015-90e8-4094-b672-8c344f5125cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:32.003439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-9a62abeb-0b84-4939-9dde-83b760c43508 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 20:21:32.003439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:32.003439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:32.003439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:32.003439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:32.003439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:32.003439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:32.003439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:32.003439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:32.003439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:32.003439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:32.003439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:32.003439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:32.003439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:32.003439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:32.003439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:32.004418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:32.004418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:32.004418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:32.004418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:32.004418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:32.004418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:32.004418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:32.004418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:32.004418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:32.004418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:32.004418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:32.004418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:32.004418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:32.004418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:32.004418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:32.004418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:32.004418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:32.004418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:32.005351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:32.005351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:32.005351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:32.005351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:32.005351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:32.005351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:32.005351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:32.005351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:32.005351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 20:21:32.005351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:32.012884 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b6c015-90e8-4094-b672-8c344f5125cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:32.012884 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b6c015-90e8-4094-b672-8c344f5125cb None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:32.013451 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 422/853] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 14 20:21:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:32.015545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62abeb-0b84-4939-9dde-83b760c43508 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:32.031710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-28b6c015-90e8-4094-b672-8c344f5125cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:32.031710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:32.031710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:32.031710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:32.031710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:32.031710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:32.031710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:32.031710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:32.031710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:32.031710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:32.031710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:32.031710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:32.031710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:32.031710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:32.031710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:32.032816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:32.032816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:32.032816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:32.032816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:32.032816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:32.032816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:32.032816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:32.032816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:32.032816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:32.032816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:32.032816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:32.032816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:32.032816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:32.032816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:32.032816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:32.032816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:32.032816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:32.033913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:32.033913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:32.033913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:32.033913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:32.039197 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 432/854] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:32.042527 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6c015-90e8-4094-b672-8c344f5125cb None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:32.094632 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:32.140655 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62abeb-0b84-4939-9dde-83b760c43508 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:32.182990 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e051e-ceaf-4b52-9c64-7adf5c574223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:32.183892 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e051e-ceaf-4b52-9c64-7adf5c574223 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:32.184275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e051e-ceaf-4b52-9c64-7adf5c574223 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:32.185239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e051e-ceaf-4b52-9c64-7adf5c574223 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:32.186529 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e051e-ceaf-4b52-9c64-7adf5c574223 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:32.186785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e051e-ceaf-4b52-9c64-7adf5c574223 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:32.188398 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e051e-ceaf-4b52-9c64-7adf5c574223 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:32.189691 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6c015-90e8-4094-b672-8c344f5125cb None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:32.200737 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:32.216386 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 423/855] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jul 14 20:21:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:32.219294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e051e-ceaf-4b52-9c64-7adf5c574223 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:32.237639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10033b-285d-407d-ac8a-4a8031a8bd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:32.238726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10033b-285d-407d-ac8a-4a8031a8bd6f None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:32.239284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10033b-285d-407d-ac8a-4a8031a8bd6f None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:32.239761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10033b-285d-407d-ac8a-4a8031a8bd6f None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:32.240691 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10033b-285d-407d-ac8a-4a8031a8bd6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:32.241198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10033b-285d-407d-ac8a-4a8031a8bd6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:32.242762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10033b-285d-407d-ac8a-4a8031a8bd6f None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:32.258389 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 433/856] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:32.263674 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10033b-285d-407d-ac8a-4a8031a8bd6f None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:32.331709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e051e-ceaf-4b52-9c64-7adf5c574223 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:32.368854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aeb902-8efc-4c22-bde9-b12f6a58077f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:32.369581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aeb902-8efc-4c22-bde9-b12f6a58077f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:32.369888 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aeb902-8efc-4c22-bde9-b12f6a58077f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:32.370164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aeb902-8efc-4c22-bde9-b12f6a58077f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:32.371882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:32.379817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10033b-285d-407d-ac8a-4a8031a8bd6f None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:32.419844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6d7d7-dd18-4c33-938f-e3d4b8f54e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:32.421036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6d7d7-dd18-4c33-938f-e3d4b8f54e2d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:32.421516 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6d7d7-dd18-4c33-938f-e3d4b8f54e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:32.421980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6d7d7-dd18-4c33-938f-e3d4b8f54e2d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:32.431001 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6d7d7-dd18-4c33-938f-e3d4b8f54e2d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:32.432760 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6d7d7-dd18-4c33-938f-e3d4b8f54e2d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:32.445995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:32.449451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-59f6d7d7-dd18-4c33-938f-e3d4b8f54e2d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 20:21:32.449451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:32.449451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:32.449451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:32.449451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:32.449451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:32.449451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:32.449451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:32.449451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:32.449451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:32.449451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:32.449451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:32.449451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:32.449451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:32.449451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:32.449451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:32.450393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:32.450393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:32.450393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:32.450393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:32.450393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:32.450393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:32.450393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:32.450393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:32.450393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:32.450393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:32.450393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:32.450393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:32.450393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:32.450393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:32.450393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:32.450393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:32.450393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:32.450393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:32.451371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:32.451371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:32.451371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:32.451371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:32.451371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:32.451371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:32.451371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:32.451371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:32.451371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 20:21:32.451371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:32.451898 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aeb902-8efc-4c22-bde9-b12f6a58077f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:32.457044 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 434/857] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:21:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:21:32.457234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aeb902-8efc-4c22-bde9-b12f6a58077f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:32.457317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71514509-3c6c-4f15-bc7f-1b35026f6ffc None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:32.457317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71514509-3c6c-4f15-bc7f-1b35026f6ffc None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:32.457317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71514509-3c6c-4f15-bc7f-1b35026f6ffc None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:32.459066 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 435/858] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:32.460265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71514509-3c6c-4f15-bc7f-1b35026f6ffc None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:32.568975 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71514509-3c6c-4f15-bc7f-1b35026f6ffc None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:32.573865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 424/859] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:21:32 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:21:32.573865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41b9cd-9bc0-46ac-9b71-582a6ec8e1e8 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:32.573865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41b9cd-9bc0-46ac-9b71-582a6ec8e1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:32.573865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41b9cd-9bc0-46ac-9b71-582a6ec8e1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:32.589528 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ab41b9cd-9bc0-46ac-9b71-582a6ec8e1e8 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:32.590761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ab41b9cd-9bc0-46ac-9b71-582a6ec8e1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:32.612570 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cebc89-b125-493d-b2f9-0f1a16f67cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:32.613264 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cebc89-b125-493d-b2f9-0f1a16f67cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:32.613546 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cebc89-b125-493d-b2f9-0f1a16f67cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:32.613819 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cebc89-b125-493d-b2f9-0f1a16f67cc4 None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:32.614661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cebc89-b125-493d-b2f9-0f1a16f67cc4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:32.614999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cebc89-b125-493d-b2f9-0f1a16f67cc4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:32.616439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cebc89-b125-493d-b2f9-0f1a16f67cc4 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:32.641369 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 436/860] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jul 14 20:21:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:32.642325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cebc89-b125-493d-b2f9-0f1a16f67cc4 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:32.664340 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:32.678986 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 425/861] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:32.678986 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41b9cd-9bc0-46ac-9b71-582a6ec8e1e8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:32.710057 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:32.763042 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cebc89-b125-493d-b2f9-0f1a16f67cc4 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:32.796495 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41b9cd-9bc0-46ac-9b71-582a6ec8e1e8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:32.801846 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673093be-55d3-4ee3-b9e1-8ffffdda4a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:32.802658 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673093be-55d3-4ee3-b9e1-8ffffdda4a9e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:32.802971 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673093be-55d3-4ee3-b9e1-8ffffdda4a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:32.803244 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673093be-55d3-4ee3-b9e1-8ffffdda4a9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:32.814777 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673093be-55d3-4ee3-b9e1-8ffffdda4a9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:32.816269 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673093be-55d3-4ee3-b9e1-8ffffdda4a9e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:32.835541 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32a200-67d8-4629-80bc-e6174cfe6b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:32.835751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-673093be-55d3-4ee3-b9e1-8ffffdda4a9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:32.835751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:32.835751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:32.835751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:32.835751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:32.835751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:32.835751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:32.835751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:32.835751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:32.835751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:32.835751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:32.835751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:32.835751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:32.835751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:32.835751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:32.836571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:32.836571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:32.836571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:32.836571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:32.836571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:32.836571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:32.836571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:32.836571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:32.836571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:32.836571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:32.836571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:32.836571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:32.836571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:32.836571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:32.836571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:32.836571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:32.836571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:32.837583 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:32.837583 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:32.837583 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:32.837583 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:32.837802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32a200-67d8-4629-80bc-e6174cfe6b2e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:32.837802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32a200-67d8-4629-80bc-e6174cfe6b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:32.837802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32a200-67d8-4629-80bc-e6174cfe6b2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:32.841818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 437/862] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:32.842771 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673093be-55d3-4ee3-b9e1-8ffffdda4a9e None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:32.909500 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:32.914407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32a200-67d8-4629-80bc-e6174cfe6b2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:32.916081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32a200-67d8-4629-80bc-e6174cfe6b2e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:32.948668 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:32.953812 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673093be-55d3-4ee3-b9e1-8ffffdda4a9e None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:32.994339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb7dd2-ac04-4434-a310-75c0b875dbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:32.995151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb7dd2-ac04-4434-a310-75c0b875dbd1 None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:32.995443 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb7dd2-ac04-4434-a310-75c0b875dbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:32.995756 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb7dd2-ac04-4434-a310-75c0b875dbd1 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:32.996443 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb7dd2-ac04-4434-a310-75c0b875dbd1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:32.998431 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb7dd2-ac04-4434-a310-75c0b875dbd1 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:33.019548 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 438/863] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 14 20:21:32 2022] POST /identity/v3/services => generated 251 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:21:33.020587 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb7dd2-ac04-4434-a310-75c0b875dbd1 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:33.043408 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 426/864] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:21:32 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:21:33.049468 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32a200-67d8-4629-80bc-e6174cfe6b2e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:33.174669 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32a200-67d8-4629-80bc-e6174cfe6b2e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:33.194158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb7dd2-ac04-4434-a310-75c0b875dbd1 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:33.208288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:33.216763 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674da29-3fa7-4a02-b3e9-d00a6172999e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:33.217567 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674da29-3fa7-4a02-b3e9-d00a6172999e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:33.217853 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674da29-3fa7-4a02-b3e9-d00a6172999e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:33.218128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674da29-3fa7-4a02-b3e9-d00a6172999e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:33.220698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:33.233207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674da29-3fa7-4a02-b3e9-d00a6172999e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:33.234682 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674da29-3fa7-4a02-b3e9-d00a6172999e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:33.240927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad9aa2-21c3-4042-9965-bba9dae82f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:33.241611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad9aa2-21c3-4042-9965-bba9dae82f66 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:33.241895 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad9aa2-21c3-4042-9965-bba9dae82f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:33.242174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad9aa2-21c3-4042-9965-bba9dae82f66 None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:33.242911 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad9aa2-21c3-4042-9965-bba9dae82f66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:33.243253 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad9aa2-21c3-4042-9965-bba9dae82f66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:33.244631 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad9aa2-21c3-4042-9965-bba9dae82f66 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:33.257443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-8674da29-3fa7-4a02-b3e9-d00a6172999e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:33.257443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:33.257443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:33.257443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:33.257443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:33.257443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:33.257443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:33.257443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:33.257443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:33.257443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:33.257443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:33.257443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:33.257443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:33.257443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:33.257443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:33.258474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:33.258474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:33.258474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:33.258474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:33.258474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:33.258474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:33.258474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:33.258474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:33.258474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:33.258474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:33.258474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:33.258474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:33.258474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:33.258474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:33.258474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:33.258474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:33.258474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:33.259407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:33.259407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:33.259407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:33.259407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:33.259407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 427/866] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:21:33.263189 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 439/866] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:33.264472 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674da29-3fa7-4a02-b3e9-d00a6172999e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:33.265882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad9aa2-21c3-4042-9965-bba9dae82f66 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:33.379000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674da29-3fa7-4a02-b3e9-d00a6172999e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:33.405912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad9aa2-21c3-4042-9965-bba9dae82f66 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:33.416185 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3aa27-81cb-4b93-947f-9d7da7076f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:33.417053 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3aa27-81cb-4b93-947f-9d7da7076f8f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:33.417340 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3aa27-81cb-4b93-947f-9d7da7076f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:33.417654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3aa27-81cb-4b93-947f-9d7da7076f8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:33.425766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3aa27-81cb-4b93-947f-9d7da7076f8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:33.427185 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3aa27-81cb-4b93-947f-9d7da7076f8f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:33.442473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-19a3aa27-81cb-4b93-947f-9d7da7076f8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:33.442473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:33.442473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:33.442473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:33.442473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:33.442473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:33.442473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:33.442473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:33.442473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:33.442473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:33.442473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:33.442473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:33.442473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:33.442473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:33.442473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:33.443461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:33.443461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:33.443461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:33.443461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:33.443461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:33.443461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:33.443461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:33.443461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:33.443461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:33.443461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:33.443461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:33.443461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:33.443461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:33.443461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:33.443461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:33.443461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:33.443461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:33.444353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:33.444353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:33.444353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:33.444353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:33.444353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:33.444353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:33.444353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:33.444353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:33.444783 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633f743-b7ba-43ea-ac79-d8efa270e139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:33.444783 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633f743-b7ba-43ea-ac79-d8efa270e139 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:33.444783 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633f743-b7ba-43ea-ac79-d8efa270e139 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:33.444783 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633f743-b7ba-43ea-ac79-d8efa270e139 None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:33.445043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633f743-b7ba-43ea-ac79-d8efa270e139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:33.445043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633f743-b7ba-43ea-ac79-d8efa270e139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:33.446967 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633f743-b7ba-43ea-ac79-d8efa270e139 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:33.448549 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 428/867] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:33.450450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3aa27-81cb-4b93-947f-9d7da7076f8f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:33.460024 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 440/868] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:33.465383 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633f743-b7ba-43ea-ac79-d8efa270e139 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:33.496423 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:33.516562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:33.562673 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3aa27-81cb-4b93-947f-9d7da7076f8f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:33.584480 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633f743-b7ba-43ea-ac79-d8efa270e139 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:33.600016 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b5ad5-07f5-4cee-a3cd-2e4815cdf546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:33.600810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b5ad5-07f5-4cee-a3cd-2e4815cdf546 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:33.600990 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b5ad5-07f5-4cee-a3cd-2e4815cdf546 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:33.601236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b5ad5-07f5-4cee-a3cd-2e4815cdf546 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:33.602037 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b5ad5-07f5-4cee-a3cd-2e4815cdf546 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:33.602416 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b5ad5-07f5-4cee-a3cd-2e4815cdf546 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:33.603828 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b5ad5-07f5-4cee-a3cd-2e4815cdf546 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:33.619676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 429/869] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:33.622834 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b5ad5-07f5-4cee-a3cd-2e4815cdf546 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:33.626083 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803c66d-3b29-4831-9b10-452081053c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:33.626983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803c66d-3b29-4831-9b10-452081053c15 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:33.627272 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803c66d-3b29-4831-9b10-452081053c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:33.627543 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803c66d-3b29-4831-9b10-452081053c15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:33.643065 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803c66d-3b29-4831-9b10-452081053c15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:33.643065 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803c66d-3b29-4831-9b10-452081053c15 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:33.655314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-d803c66d-3b29-4831-9b10-452081053c15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:33.655314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:33.655314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:33.655314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:33.655314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:33.655314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:33.655314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:33.655314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:33.655314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:33.655314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:33.655314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:33.655314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:33.655314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:33.655314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:33.655314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:33.656470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:33.656470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:33.656470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:33.656470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:33.656470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:33.656470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:33.656470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:33.656470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:33.656470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:33.656470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:33.656470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:33.656470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:33.656470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:33.656470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:33.656470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:33.656470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:33.656470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:33.657571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:33.657571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:33.657571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:33.657571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:33.657571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:33.657571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:33.657571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:33.657571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:33.661331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 441/870] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:33.662228 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803c66d-3b29-4831-9b10-452081053c15 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:33.734639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:33.738167 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b5ad5-07f5-4cee-a3cd-2e4815cdf546 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:33.768292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:33.771965 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803c66d-3b29-4831-9b10-452081053c15 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:33.779659 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b147fb4-6f09-4eb1-9722-483aaa52b4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:33.780404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b147fb4-6f09-4eb1-9722-483aaa52b4ef None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:33.780755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b147fb4-6f09-4eb1-9722-483aaa52b4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:33.781097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b147fb4-6f09-4eb1-9722-483aaa52b4ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:33.781992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b147fb4-6f09-4eb1-9722-483aaa52b4ef None admin] RBAC: Adding query filter params (role.id=ea49c36b4a7f4cfb9048065638d458af, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=4f75b8b484384554ad59aa20535875e9) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:33.782602 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b147fb4-6f09-4eb1-9722-483aaa52b4ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:33.783934 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b147fb4-6f09-4eb1-9722-483aaa52b4ef None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:33.794976 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 430/871] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:21:33 2022] GET /identity/v3/role_assignments?user.id=4f75b8b484384554ad59aa20535875e9&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=ea49c36b4a7f4cfb9048065638d458af => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:33.794976 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d9bb2-16cc-4122-b949-a89665141bbd None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:33.794976 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d9bb2-16cc-4122-b949-a89665141bbd None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:33.794976 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d9bb2-16cc-4122-b949-a89665141bbd None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:33.794976 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 431/872] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:33.800753 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd37d8-cf6d-4976-93fa-c0d6a6faf770 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:33.801416 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd37d8-cf6d-4976-93fa-c0d6a6faf770 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:33.801416 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd37d8-cf6d-4976-93fa-c0d6a6faf770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:33.821310 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-0fdd37d8-cf6d-4976-93fa-c0d6a6faf770 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:33.822582 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0fdd37d8-cf6d-4976-93fa-c0d6a6faf770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:33.827379 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acf186-00ad-4d87-b0d0-275da006c514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:33.828901 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acf186-00ad-4d87-b0d0-275da006c514 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:33.829418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acf186-00ad-4d87-b0d0-275da006c514 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:33.829872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acf186-00ad-4d87-b0d0-275da006c514 None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:33.830984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acf186-00ad-4d87-b0d0-275da006c514 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:33.831516 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acf186-00ad-4d87-b0d0-275da006c514 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:33.833405 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acf186-00ad-4d87-b0d0-275da006c514 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:33.852882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 442/873] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:33.854199 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acf186-00ad-4d87-b0d0-275da006c514 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:33.919407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 432/874] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 20:21:33.938384 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd37d8-cf6d-4976-93fa-c0d6a6faf770 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:33.971740 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acf186-00ad-4d87-b0d0-275da006c514 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:34.016402 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddecabea-1158-494e-b087-4053143243c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:34.017085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddecabea-1158-494e-b087-4053143243c6 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:34.017551 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddecabea-1158-494e-b087-4053143243c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:34.017829 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddecabea-1158-494e-b087-4053143243c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:34.020068 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddecabea-1158-494e-b087-4053143243c6 None admin] RBAC: Adding query filter params (role.id=5ca112885467422684e134cb0f94865a, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=acc23233929144a5b7b3a1733da19828) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:34.020068 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddecabea-1158-494e-b087-4053143243c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:34.021342 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddecabea-1158-494e-b087-4053143243c6 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:34.029857 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 443/875] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:21:33 2022] GET /identity/v3/role_assignments?user.id=acc23233929144a5b7b3a1733da19828&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=5ca112885467422684e134cb0f94865a => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:34.068018 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:34.072370 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd37d8-cf6d-4976-93fa-c0d6a6faf770 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:34.116434 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af65c66-9d6d-408d-b6ec-e148695b6304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:34.117128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af65c66-9d6d-408d-b6ec-e148695b6304 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:34.117414 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af65c66-9d6d-408d-b6ec-e148695b6304 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:34.117692 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af65c66-9d6d-408d-b6ec-e148695b6304 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:34.118453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af65c66-9d6d-408d-b6ec-e148695b6304 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:34.119913 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af65c66-9d6d-408d-b6ec-e148695b6304 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:34.139161 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 433/876] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 14 20:21:33 2022] POST /identity/v3/services => generated 269 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:21:35.329921 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af8fc8-d737-4076-a627-fc79ed0d4d3d None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:35.330992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af8fc8-d737-4076-a627-fc79ed0d4d3d None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:35.331488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af8fc8-d737-4076-a627-fc79ed0d4d3d None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:35.333311 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 444/877] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:35.338828 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e1ee8-4f69-4091-894c-58b74d77f6ad None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:35.339676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e1ee8-4f69-4091-894c-58b74d77f6ad None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:35.340191 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e1ee8-4f69-4091-894c-58b74d77f6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:35.358196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f38e1ee8-4f69-4091-894c-58b74d77f6ad None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:35.359751 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f38e1ee8-4f69-4091-894c-58b74d77f6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:35.436358 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 434/878] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:35.458448 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af8fc8-d737-4076-a627-fc79ed0d4d3d None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:35.486070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:35.567655 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af8fc8-d737-4076-a627-fc79ed0d4d3d None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:35.603175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f10e0-9080-49fa-9973-348031e069b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:35.604325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f10e0-9080-49fa-9973-348031e069b4 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:35.604804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f10e0-9080-49fa-9973-348031e069b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:35.605301 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f10e0-9080-49fa-9973-348031e069b4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:35.606780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f10e0-9080-49fa-9973-348031e069b4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:35.608368 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f10e0-9080-49fa-9973-348031e069b4 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:35.621004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-281f10e0-9080-49fa-9973-348031e069b4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 20:21:35.621004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:35.621004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:35.621004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:35.621004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:35.621004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:35.621004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:35.621004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:35.621004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 20:21:35.621004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:35.621004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:35.621004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:35.621004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:35.621004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:35.621004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:35.621004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:35.621004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:35.622124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:35.622124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:35.622124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:35.622124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:35.622124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:35.622124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:35.622124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:35.622124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:35.622124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:35.622124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:35.622124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:35.622124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:35.622124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:35.622124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:35.622124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:35.622124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:35.622977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:35.622977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:35.622977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:35.622977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:35.622977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:35.622977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:35.622977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:35.622977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:35.622977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:35.622977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:35.622977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:35.622977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:35.622977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 20:21:35.622977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:35.625882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 445/879] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:21:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:35.634252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e1ee8-4f69-4091-894c-58b74d77f6ad None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:35.731511 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:35.747470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e1ee8-4f69-4091-894c-58b74d77f6ad None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:35.786398 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdc005-850f-47dd-9022-e7bb210fbaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:35.787357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdc005-850f-47dd-9022-e7bb210fbaef None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:35.787791 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdc005-850f-47dd-9022-e7bb210fbaef None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:35.788218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdc005-850f-47dd-9022-e7bb210fbaef None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:35.789110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdc005-850f-47dd-9022-e7bb210fbaef None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:35.789581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdc005-850f-47dd-9022-e7bb210fbaef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:35.791143 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdc005-850f-47dd-9022-e7bb210fbaef None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:35.797127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 435/880] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:21:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:35.802547 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f10e0-9080-49fa-9973-348031e069b4 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:35.900245 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:35.915653 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f10e0-9080-49fa-9973-348031e069b4 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:35.952242 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2710f54-154c-46b9-b27a-fc1241ce1e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:35.952916 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2710f54-154c-46b9-b27a-fc1241ce1e56 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:35.953196 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2710f54-154c-46b9-b27a-fc1241ce1e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:35.953466 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2710f54-154c-46b9-b27a-fc1241ce1e56 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:35.954173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2710f54-154c-46b9-b27a-fc1241ce1e56 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:35.954474 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2710f54-154c-46b9-b27a-fc1241ce1e56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:35.955866 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2710f54-154c-46b9-b27a-fc1241ce1e56 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:35.961520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 446/881] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:21:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:35.968232 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdc005-850f-47dd-9022-e7bb210fbaef None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:36.038903 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bdc89-04e2-4756-9f6a-2155a13abf69 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:36.039214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bdc89-04e2-4756-9f6a-2155a13abf69 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:36.039489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bdc89-04e2-4756-9f6a-2155a13abf69 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:36.041136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 447/882] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:21:36.046455 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3d0b3-b678-4260-b55b-6a01dca66b4e None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:36.046811 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3d0b3-b678-4260-b55b-6a01dca66b4e None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:36.047041 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3d0b3-b678-4260-b55b-6a01dca66b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:36.065670 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-89e3d0b3-b678-4260-b55b-6a01dca66b4e None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:36.066862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-89e3d0b3-b678-4260-b55b-6a01dca66b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:36.081438 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdc005-850f-47dd-9022-e7bb210fbaef None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:36.118395 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eac0d2-2132-45ca-a5ce-5923eeb3a5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:36.119443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eac0d2-2132-45ca-a5ce-5923eeb3a5ae None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:36.119927 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eac0d2-2132-45ca-a5ce-5923eeb3a5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:36.120432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eac0d2-2132-45ca-a5ce-5923eeb3a5ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:36.121415 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eac0d2-2132-45ca-a5ce-5923eeb3a5ae None admin] RBAC: Adding query filter params (interface=public, service_id=33465a7c3a2449b3bd420a8b8032df6d, region_id=RegionOne) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:36.122482 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eac0d2-2132-45ca-a5ce-5923eeb3a5ae None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:36.124279 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eac0d2-2132-45ca-a5ce-5923eeb3a5ae None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:36.130220 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 436/883] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 14 20:21:35 2022] GET /identity/v3/endpoints?service_id=33465a7c3a2449b3bd420a8b8032df6d&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:36.135786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eac0d2-2132-45ca-a5ce-5923eeb3a5ae None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:36.148074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:36.154915 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 448/884] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 20:21:36.156921 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4582e4e-d792-4311-ae45-b76770b00ce7 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:36.157249 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4582e4e-d792-4311-ae45-b76770b00ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:36.157476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4582e4e-d792-4311-ae45-b76770b00ce7 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:36.158973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 449/885] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:36.164689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d240a-6bc2-4003-8c91-92e168c9a5f4 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:36.165278 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d240a-6bc2-4003-8c91-92e168c9a5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:36.165776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d240a-6bc2-4003-8c91-92e168c9a5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:36.183826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-f80d240a-6bc2-4003-8c91-92e168c9a5f4 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:36.185554 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f80d240a-6bc2-4003-8c91-92e168c9a5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:36.255626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eac0d2-2132-45ca-a5ce-5923eeb3a5ae None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:36.260868 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 450/886] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 20:21:36.263860 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d240a-6bc2-4003-8c91-92e168c9a5f4 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:36.283470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:36.296393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389f1c0-00f1-40a4-9fb2-9b32457aaf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:36.297228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389f1c0-00f1-40a4-9fb2-9b32457aaf2b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:36.297515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389f1c0-00f1-40a4-9fb2-9b32457aaf2b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:36.297794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389f1c0-00f1-40a4-9fb2-9b32457aaf2b None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:36.298490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389f1c0-00f1-40a4-9fb2-9b32457aaf2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:36.299954 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389f1c0-00f1-40a4-9fb2-9b32457aaf2b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:36.306054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 437/887] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 14 20:21:36 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:21:36.320037 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389f1c0-00f1-40a4-9fb2-9b32457aaf2b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:36.399316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d240a-6bc2-4003-8c91-92e168c9a5f4 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:36.405482 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:36.442745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389f1c0-00f1-40a4-9fb2-9b32457aaf2b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:36.443030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefc95e-9966-4e08-9a81-2e62b4838c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:36.443868 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefc95e-9966-4e08-9a81-2e62b4838c4d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:36.444157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefc95e-9966-4e08-9a81-2e62b4838c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:36.444434 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefc95e-9966-4e08-9a81-2e62b4838c4d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:36.456741 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caefc95e-9966-4e08-9a81-2e62b4838c4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:36.465156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-caefc95e-9966-4e08-9a81-2e62b4838c4d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 20:21:36.465156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:36.465156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:36.465156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:36.465156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:36.465156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:36.465156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:36.465156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:36.465156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:36.465156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:36.465156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:36.465156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:36.465156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:36.465156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:36.465156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:36.465156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:36.466153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:36.466153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:36.466153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:36.466153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:36.466153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:36.466153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:36.466153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:36.466153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:36.466153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:36.466153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:36.466153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:36.466153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:36.466153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:36.466153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:36.466153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:36.466153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:36.470478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:36.470478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:36.470478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:36.470478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:36.470478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:36.470478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:36.470478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:36.470478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:36.470478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:36.470478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:36.470478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 20:21:36.470478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:36.470478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 451/888] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jul 14 20:21:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:36.470478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220987d-ca10-406d-9b27-6c7126cc0b68 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:36.471672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220987d-ca10-406d-9b27-6c7126cc0b68 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:36.471672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220987d-ca10-406d-9b27-6c7126cc0b68 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:36.471672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 452/889] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:36.472841 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220987d-ca10-406d-9b27-6c7126cc0b68 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:36.491648 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2cf10-663c-4d98-9555-303138a555a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:36.492470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2cf10-663c-4d98-9555-303138a555a2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:36.492963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2cf10-663c-4d98-9555-303138a555a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:36.493250 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2cf10-663c-4d98-9555-303138a555a2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:36.493910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2cf10-663c-4d98-9555-303138a555a2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:36.495319 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2cf10-663c-4d98-9555-303138a555a2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:36.508396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-53f2cf10-663c-4d98-9555-303138a555a2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 20:21:36.508396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:36.508396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:36.508396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:36.508396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:36.508396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:36.508396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:36.508396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:36.508396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 20:21:36.508396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:36.508396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:36.508396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:36.508396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:36.508396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:36.508396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:36.508396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:36.508396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:36.509610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:36.509610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:36.509610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:36.509610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:36.509610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:36.509610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:36.509610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:36.509610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:36.509610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:36.509610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:36.509610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:36.509610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:36.509610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:36.509610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:36.509610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:36.509610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:36.510683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:36.510683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:36.510683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:36.510683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:36.510683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:36.510683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:36.510683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:36.510683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:36.510683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:36.510683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:36.510683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:36.510683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:36.510683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 20:21:36.510683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:36.510683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 438/890] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 14 20:21:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:36.511578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd27429-df7d-4e9c-a59f-6d950fd415bd None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:36.511578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd27429-df7d-4e9c-a59f-6d950fd415bd None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:36.512034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd27429-df7d-4e9c-a59f-6d950fd415bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:36.533450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-0dd27429-df7d-4e9c-a59f-6d950fd415bd None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:36.534659 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0dd27429-df7d-4e9c-a59f-6d950fd415bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:36.561346 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:36.598564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220987d-ca10-406d-9b27-6c7126cc0b68 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:36.614063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 439/891] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:36.615250 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd27429-df7d-4e9c-a59f-6d950fd415bd None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:36.654855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b5fbc-3278-4be4-84d7-9eef249802e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:36.654855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b5fbc-3278-4be4-84d7-9eef249802e9 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:36.655373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b5fbc-3278-4be4-84d7-9eef249802e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:36.655695 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b5fbc-3278-4be4-84d7-9eef249802e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:36.656371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b5fbc-3278-4be4-84d7-9eef249802e9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:36.656680 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b5fbc-3278-4be4-84d7-9eef249802e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:36.658090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b5fbc-3278-4be4-84d7-9eef249802e9 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:36.668486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 453/892] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 14 20:21:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:36.669960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b5fbc-3278-4be4-84d7-9eef249802e9 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:36.703097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:36.758964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd27429-df7d-4e9c-a59f-6d950fd415bd None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:36.814846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8ee70-bbd5-4da4-af43-b58780a2e48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:36.815964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8ee70-bbd5-4da4-af43-b58780a2e48a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:36.816354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8ee70-bbd5-4da4-af43-b58780a2e48a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:36.816644 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8ee70-bbd5-4da4-af43-b58780a2e48a None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:36.817382 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8ee70-bbd5-4da4-af43-b58780a2e48a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:36.817747 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8ee70-bbd5-4da4-af43-b58780a2e48a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:36.819198 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8ee70-bbd5-4da4-af43-b58780a2e48a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:36.825432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 440/893] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 14 20:21:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:36.831271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8ee70-bbd5-4da4-af43-b58780a2e48a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:36.831849 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b5fbc-3278-4be4-84d7-9eef249802e9 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:36.888078 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e31be-4601-433b-8dde-b077232b492f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:36.889226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e31be-4601-433b-8dde-b077232b492f None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:36.889500 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e31be-4601-433b-8dde-b077232b492f None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:36.889781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e31be-4601-433b-8dde-b077232b492f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:36.890443 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e31be-4601-433b-8dde-b077232b492f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:36.891842 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e31be-4601-433b-8dde-b077232b492f None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:36.893858 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:36.906507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-dc7e31be-4601-433b-8dde-b077232b492f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 20:21:36.906507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:36.906507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:36.906507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:36.906507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:36.906507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:36.906507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:36.906507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:36.906507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 20:21:36.906507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:36.906507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:36.906507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:36.906507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:36.906507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:36.906507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:36.906507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:36.907544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:36.907544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:36.907544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:36.907544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:36.907544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:36.907544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:36.907544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:36.907544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:36.907544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:36.907544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:36.907544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:36.907544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:36.907544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:36.907544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:36.907544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:36.907544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:36.908370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:36.908370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:36.908370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:36.908370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:36.908370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:36.908370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:36.908370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:36.908370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:36.908370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:36.908370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:36.908370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:36.908370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:36.908370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:36.908370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 20:21:36.908370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:36.908370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 454/894] 10.222.0.52 () {56 vars in 1132 bytes} [Thu Jul 14 20:21:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:36.910331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e31be-4601-433b-8dde-b077232b492f None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:36.985134 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8ee70-bbd5-4da4-af43-b58780a2e48a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:37.018351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e31be-4601-433b-8dde-b077232b492f None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:37.033354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:37.041133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87122d1-7918-4427-986e-ce5e4394963a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:37.041947 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87122d1-7918-4427-986e-ce5e4394963a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:37.042292 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87122d1-7918-4427-986e-ce5e4394963a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:37.042575 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87122d1-7918-4427-986e-ce5e4394963a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:37.054452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87122d1-7918-4427-986e-ce5e4394963a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:37.055941 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87122d1-7918-4427-986e-ce5e4394963a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:37.064812 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f73b75-136f-4259-8905-1a1e6baa7b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:37.065485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f73b75-136f-4259-8905-1a1e6baa7b35 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:37.066224 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f73b75-136f-4259-8905-1a1e6baa7b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:37.066224 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f73b75-136f-4259-8905-1a1e6baa7b35 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:37.066743 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f73b75-136f-4259-8905-1a1e6baa7b35 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:37.067175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f73b75-136f-4259-8905-1a1e6baa7b35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:37.068447 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f73b75-136f-4259-8905-1a1e6baa7b35 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:37.083108 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 455/895] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 14 20:21:36 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:37.083108 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f73b75-136f-4259-8905-1a1e6baa7b35 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:37.083366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-f87122d1-7918-4427-986e-ce5e4394963a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:37.083366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:37.083366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:37.083366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:37.083366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:37.083366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:37.083366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:37.083366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:37.083366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:37.083366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:37.083366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:37.083366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:37.083366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:37.083366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:37.083366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:37.084189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:37.084189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:37.084189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:37.084189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:37.084189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:37.084189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:37.084189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:37.084189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:37.084189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:37.084189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:37.084189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:37.084189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:37.084189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:37.084189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:37.084189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:37.084189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:37.084189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:37.085095 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:37.085095 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:37.085095 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:37.085095 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:37.085095 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 441/896] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:37.085095 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87122d1-7918-4427-986e-ce5e4394963a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:37.170547 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:37.214438 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87122d1-7918-4427-986e-ce5e4394963a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:37.233007 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f73b75-136f-4259-8905-1a1e6baa7b35 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:37.259571 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd8aad-4b9f-4927-9293-029b648372d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:37.260532 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd8aad-4b9f-4927-9293-029b648372d0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:37.260814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd8aad-4b9f-4927-9293-029b648372d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:37.261091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd8aad-4b9f-4927-9293-029b648372d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:37.264691 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd8aad-4b9f-4927-9293-029b648372d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:37.264691 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd8aad-4b9f-4927-9293-029b648372d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:37.264691 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd8aad-4b9f-4927-9293-029b648372d0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:37.277168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 442/897] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:37.279420 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f432ba-1cf5-4eff-b7a7-7b308f9f56cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:37.280098 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f432ba-1cf5-4eff-b7a7-7b308f9f56cf None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:37.280448 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f432ba-1cf5-4eff-b7a7-7b308f9f56cf None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:37.280732 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f432ba-1cf5-4eff-b7a7-7b308f9f56cf None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:37.281455 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f432ba-1cf5-4eff-b7a7-7b308f9f56cf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:37.281766 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f432ba-1cf5-4eff-b7a7-7b308f9f56cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:37.283209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f432ba-1cf5-4eff-b7a7-7b308f9f56cf None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:37.287988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd8aad-4b9f-4927-9293-029b648372d0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:37.288570 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 456/898] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 14 20:21:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:37.295010 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f432ba-1cf5-4eff-b7a7-7b308f9f56cf None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:37.349457 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:37.409391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f432ba-1cf5-4eff-b7a7-7b308f9f56cf None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:37.427297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd8aad-4b9f-4927-9293-029b648372d0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:37.439347 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:37.448001 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6a6af-8c59-4243-83f5-9aeaadaa9e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:37.448742 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6a6af-8c59-4243-83f5-9aeaadaa9e80 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:37.449130 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6a6af-8c59-4243-83f5-9aeaadaa9e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:37.449509 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6a6af-8c59-4243-83f5-9aeaadaa9e80 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:37.450882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6a6af-8c59-4243-83f5-9aeaadaa9e80 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:37.451327 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6a6af-8c59-4243-83f5-9aeaadaa9e80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:37.453010 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6a6af-8c59-4243-83f5-9aeaadaa9e80 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:37.463708 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 457/899] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 14 20:21:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:37.466433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6a6af-8c59-4243-83f5-9aeaadaa9e80 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:37.469615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd75e9f-4aa5-489b-958b-0b2eeb5f83f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:37.470528 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd75e9f-4aa5-489b-958b-0b2eeb5f83f7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:37.470811 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd75e9f-4aa5-489b-958b-0b2eeb5f83f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:37.471113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd75e9f-4aa5-489b-958b-0b2eeb5f83f7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:37.480409 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd75e9f-4aa5-489b-958b-0b2eeb5f83f7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:37.482029 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd75e9f-4aa5-489b-958b-0b2eeb5f83f7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:37.498304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-cfd75e9f-4aa5-489b-958b-0b2eeb5f83f7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 20:21:37.498304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:37.498304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:37.498304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:37.498304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:37.498304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:37.498304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:37.498304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:37.498304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:37.498304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:37.498304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:37.498304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:37.498304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:37.498304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:37.498304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:37.498304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:37.499326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:37.499326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:37.499326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:37.499326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:37.499326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:37.499326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:37.499326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:37.499326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:37.499326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:37.499326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:37.499326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:37.499326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:37.499326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:37.499326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:37.499326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:37.499326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:37.499326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:37.499326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:37.500304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:37.500304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:37.500304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:37.500304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:37.500304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:37.500304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:37.500304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:37.500304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:37.500304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 20:21:37.500304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:37.500304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 443/900] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 14 20:21:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:37.505335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd75e9f-4aa5-489b-958b-0b2eeb5f83f7 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:37.591486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6a6af-8c59-4243-83f5-9aeaadaa9e80 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:37.632940 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a67ff-2f31-4034-b740-728bb4bdf916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:37.633619 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a67ff-2f31-4034-b740-728bb4bdf916 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:37.633905 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a67ff-2f31-4034-b740-728bb4bdf916 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:37.634182 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a67ff-2f31-4034-b740-728bb4bdf916 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:37.635151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a67ff-2f31-4034-b740-728bb4bdf916 None admin] RBAC: Adding query filter params (interface=public, service_id=1b93a0463fb54c02bc419a7cd516bc5a, region_id=RegionOne) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:37.635469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a67ff-2f31-4034-b740-728bb4bdf916 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:37.636856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a67ff-2f31-4034-b740-728bb4bdf916 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:37.650062 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 458/901] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 14 20:21:37 2022] GET /identity/v3/endpoints?service_id=1b93a0463fb54c02bc419a7cd516bc5a&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:37.651324 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a67ff-2f31-4034-b740-728bb4bdf916 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:37.653242 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd75e9f-4aa5-489b-958b-0b2eeb5f83f7 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:37.655491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:37.701947 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9593a-a7bb-40ca-b36d-bdd5a53b0055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:37.703315 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9593a-a7bb-40ca-b36d-bdd5a53b0055 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:37.704121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9593a-a7bb-40ca-b36d-bdd5a53b0055 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:37.704658 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9593a-a7bb-40ca-b36d-bdd5a53b0055 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:37.705884 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9593a-a7bb-40ca-b36d-bdd5a53b0055 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:37.706790 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9593a-a7bb-40ca-b36d-bdd5a53b0055 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:37.708931 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9593a-a7bb-40ca-b36d-bdd5a53b0055 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:37.716077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:37.740108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 444/902] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jul 14 20:21:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:37.741475 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9593a-a7bb-40ca-b36d-bdd5a53b0055 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:37.769724 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a67ff-2f31-4034-b740-728bb4bdf916 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:37.807893 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52bffb2-f71a-4431-8797-8c6a1435a9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:37.808695 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52bffb2-f71a-4431-8797-8c6a1435a9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:37.808953 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52bffb2-f71a-4431-8797-8c6a1435a9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:37.809778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52bffb2-f71a-4431-8797-8c6a1435a9e5 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:37.810486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52bffb2-f71a-4431-8797-8c6a1435a9e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:37.812083 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52bffb2-f71a-4431-8797-8c6a1435a9e5 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:37.823964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 459/903] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 14 20:21:37 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:21:37.860681 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9593a-a7bb-40ca-b36d-bdd5a53b0055 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:37.901639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba578c-1d58-4b99-a391-4f1018d3bc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:37.902314 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba578c-1d58-4b99-a391-4f1018d3bc5d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:37.902601 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba578c-1d58-4b99-a391-4f1018d3bc5d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:37.903000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba578c-1d58-4b99-a391-4f1018d3bc5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:37.937780 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:37.984298 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba578c-1d58-4b99-a391-4f1018d3bc5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:37.985718 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba578c-1d58-4b99-a391-4f1018d3bc5d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:38.096518 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 445/904] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:21:37 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:21:38.102847 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52bffb2-f71a-4431-8797-8c6a1435a9e5 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:38.219441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52bffb2-f71a-4431-8797-8c6a1435a9e5 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:38.238464 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:38.265676 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db488873-07e0-41c1-88c2-3fd140686fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:38.266477 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db488873-07e0-41c1-88c2-3fd140686fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:38.266757 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db488873-07e0-41c1-88c2-3fd140686fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:38.267077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db488873-07e0-41c1-88c2-3fd140686fc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:38.279033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db488873-07e0-41c1-88c2-3fd140686fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:38.280502 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db488873-07e0-41c1-88c2-3fd140686fc0 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:38.299003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-db488873-07e0-41c1-88c2-3fd140686fc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:38.299003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:38.299003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:38.299003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:38.299003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:38.299003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:38.299003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:38.299003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:38.299003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:38.299003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:38.299003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:38.299003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:38.299003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:38.299003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:38.299003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:38.300008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:38.300008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:38.300008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:38.300008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:38.300008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:38.300008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:38.300008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:38.300008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:38.300008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:38.300008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:38.300008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:38.300008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:38.300008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:38.300008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:38.300008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:38.300008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:38.300008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:38.300936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:38.300936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:38.300936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:38.300936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:38.300936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 460/905] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:38.307507 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba578c-1d58-4b99-a391-4f1018d3bc5d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:38.375115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:38.420675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba578c-1d58-4b99-a391-4f1018d3bc5d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:38.456584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1d35a-754b-4346-925f-d41a1378749b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:38.458129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1d35a-754b-4346-925f-d41a1378749b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:38.458670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1d35a-754b-4346-925f-d41a1378749b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:38.459180 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1d35a-754b-4346-925f-d41a1378749b None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:38.460125 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1d35a-754b-4346-925f-d41a1378749b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:38.460643 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1d35a-754b-4346-925f-d41a1378749b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:38.462473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1d35a-754b-4346-925f-d41a1378749b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:38.476468 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 446/906] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:38.482961 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db488873-07e0-41c1-88c2-3fd140686fc0 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:38.534261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c8524-958f-4522-b82f-94d9517de76b None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:38.534893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c8524-958f-4522-b82f-94d9517de76b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:38.535509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c8524-958f-4522-b82f-94d9517de76b None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:38.537493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 447/907] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:38.544106 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3e623-c531-4349-bac9-81793f7d26b0 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:38.544412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3e623-c531-4349-bac9-81793f7d26b0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:38.544693 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3e623-c531-4349-bac9-81793f7d26b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:38.566124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-54c3e623-c531-4349-bac9-81793f7d26b0 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:38.567677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-54c3e623-c531-4349-bac9-81793f7d26b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:38.608047 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db488873-07e0-41c1-88c2-3fd140686fc0 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:38.654111 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22feeaa-a844-46ce-b0fd-3d3b50561e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:38.655215 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22feeaa-a844-46ce-b0fd-3d3b50561e4f None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:38.655809 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22feeaa-a844-46ce-b0fd-3d3b50561e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:38.656245 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22feeaa-a844-46ce-b0fd-3d3b50561e4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:38.665865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22feeaa-a844-46ce-b0fd-3d3b50561e4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:38.667614 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22feeaa-a844-46ce-b0fd-3d3b50561e4f None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:38.671089 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 448/908] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:38.687986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-d22feeaa-a844-46ce-b0fd-3d3b50561e4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:38.687986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:38.687986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:38.687986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:38.687986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:38.687986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:38.687986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:38.687986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:38.687986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:38.687986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:38.687986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:38.687986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:38.687986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:38.687986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:38.687986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:38.689201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:38.689201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:38.689201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:38.689201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:38.689201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:38.689201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:38.689201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:38.689201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:38.689201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:38.689201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:38.689201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:38.689201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:38.689201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:38.689201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:38.689201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:38.689201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:38.689201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:38.690183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:38.690183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:38.690183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:38.690183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:38.690183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:38.690183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:38.690183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:38.690183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:38.692407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 461/909] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:38.698202 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3e623-c531-4349-bac9-81793f7d26b0 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:38.700641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22feeaa-a844-46ce-b0fd-3d3b50561e4f None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:38.736533 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:38.773734 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:38.821132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22feeaa-a844-46ce-b0fd-3d3b50561e4f None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:38.829957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3e623-c531-4349-bac9-81793f7d26b0 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:38.861169 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0330b-2e53-4799-81be-2e672d555063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:38.862337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0330b-2e53-4799-81be-2e672d555063 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:38.862966 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0330b-2e53-4799-81be-2e672d555063 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:38.863463 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0330b-2e53-4799-81be-2e672d555063 None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:38.864670 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0330b-2e53-4799-81be-2e672d555063 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:38.865243 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0330b-2e53-4799-81be-2e672d555063 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:38.867143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0330b-2e53-4799-81be-2e672d555063 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:38.867742 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2211a-2045-432f-bcc2-5bb35c77de0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:38.868558 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2211a-2045-432f-bcc2-5bb35c77de0a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:38.868846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2211a-2045-432f-bcc2-5bb35c77de0a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:38.869127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2211a-2045-432f-bcc2-5bb35c77de0a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:38.869790 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2211a-2045-432f-bcc2-5bb35c77de0a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:38.871548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2211a-2045-432f-bcc2-5bb35c77de0a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:38.882852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 462/910] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:38.889859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-27f2211a-2045-432f-bcc2-5bb35c77de0a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 20:21:38.889859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:38.889859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:38.889859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:38.889859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:38.889859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:38.889859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:38.889859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:38.889859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 20:21:38.889859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:38.889859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:38.889859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:38.889859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:38.889859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:38.889859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:38.889859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:38.889859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:38.891014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:38.891014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:38.891014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:38.891014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:38.891014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:38.891014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:38.891014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:38.891014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:38.891014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:38.891014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:38.891014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:38.891014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:38.891014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:38.891014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:38.891014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:38.891014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:38.891881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:38.891881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:38.891881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:38.891881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:38.891881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:38.891881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:38.891881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:38.891881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:38.891881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:38.891881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:38.891881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:38.891881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:38.891881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 20:21:38.891881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:38.892628 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0330b-2e53-4799-81be-2e672d555063 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:38.894291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 449/911] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:21:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:38.899795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2211a-2045-432f-bcc2-5bb35c77de0a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:39.032855 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2211a-2045-432f-bcc2-5bb35c77de0a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:39.033526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:39.046828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0330b-2e53-4799-81be-2e672d555063 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:39.050581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:39.083251 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4469b23-e42f-4070-a7bc-bc6638c24c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:39.084328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4469b23-e42f-4070-a7bc-bc6638c24c6d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:39.084863 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4469b23-e42f-4070-a7bc-bc6638c24c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:39.085242 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4469b23-e42f-4070-a7bc-bc6638c24c6d None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:39.086102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4469b23-e42f-4070-a7bc-bc6638c24c6d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:39.086519 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4469b23-e42f-4070-a7bc-bc6638c24c6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:39.088062 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4469b23-e42f-4070-a7bc-bc6638c24c6d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:39.094029 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 450/912] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:21:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:39.104595 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4469b23-e42f-4070-a7bc-bc6638c24c6d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:39.108863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18be83-0c82-44d2-b8bd-20402aef7aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:39.110290 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18be83-0c82-44d2-b8bd-20402aef7aed None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:39.111162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18be83-0c82-44d2-b8bd-20402aef7aed None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:39.111949 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18be83-0c82-44d2-b8bd-20402aef7aed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:39.113742 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18be83-0c82-44d2-b8bd-20402aef7aed None admin] RBAC: Adding query filter params (role.id=f932ef767ec247d8ad3823e16f1b0306, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=4f75b8b484384554ad59aa20535875e9) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:39.114559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18be83-0c82-44d2-b8bd-20402aef7aed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:39.117526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18be83-0c82-44d2-b8bd-20402aef7aed None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:39.129798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 463/913] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:21:38 2022] GET /identity/v3/role_assignments?user.id=4f75b8b484384554ad59aa20535875e9&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=f932ef767ec247d8ad3823e16f1b0306 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:39.180442 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1366c6-84ed-4baf-9d5b-2affe38d7596 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:39.181425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1366c6-84ed-4baf-9d5b-2affe38d7596 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:39.182219 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1366c6-84ed-4baf-9d5b-2affe38d7596 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:39.186535 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 464/914] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:39.193287 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f98560-dcc7-4602-a112-9df4f578fd06 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:39.193678 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f98560-dcc7-4602-a112-9df4f578fd06 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:39.193864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f98560-dcc7-4602-a112-9df4f578fd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:39.212393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-24f98560-dcc7-4602-a112-9df4f578fd06 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:39.213623 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-24f98560-dcc7-4602-a112-9df4f578fd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:39.288578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4469b23-e42f-4070-a7bc-bc6638c24c6d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:39.316879 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 465/915] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 20:21:39.335652 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692b2ee-3142-416c-bb6e-09fff9d3aa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:39.336668 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692b2ee-3142-416c-bb6e-09fff9d3aa64 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:39.337174 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692b2ee-3142-416c-bb6e-09fff9d3aa64 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:39.337667 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692b2ee-3142-416c-bb6e-09fff9d3aa64 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:39.338629 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692b2ee-3142-416c-bb6e-09fff9d3aa64 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:39.340104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f98560-dcc7-4602-a112-9df4f578fd06 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:39.343217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692b2ee-3142-416c-bb6e-09fff9d3aa64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:39.344878 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692b2ee-3142-416c-bb6e-09fff9d3aa64 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:39.351830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 451/916] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 14 20:21:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:39.357344 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692b2ee-3142-416c-bb6e-09fff9d3aa64 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:39.404260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:39.434850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:39.453730 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f98560-dcc7-4602-a112-9df4f578fd06 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:39.469933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692b2ee-3142-416c-bb6e-09fff9d3aa64 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:39.492277 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1620296-5ca7-4aba-8ca9-f3b76ed5627b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:39.493661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1620296-5ca7-4aba-8ca9-f3b76ed5627b None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:39.494168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1620296-5ca7-4aba-8ca9-f3b76ed5627b None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:39.494642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1620296-5ca7-4aba-8ca9-f3b76ed5627b None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:39.495655 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1620296-5ca7-4aba-8ca9-f3b76ed5627b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:39.497817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1620296-5ca7-4aba-8ca9-f3b76ed5627b None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:39.508140 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22c36a-fba9-45cd-860a-6567e43f7688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:39.509073 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22c36a-fba9-45cd-860a-6567e43f7688 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:39.509377 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22c36a-fba9-45cd-860a-6567e43f7688 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:39.509910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22c36a-fba9-45cd-860a-6567e43f7688 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:39.510551 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22c36a-fba9-45cd-860a-6567e43f7688 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:39.512056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22c36a-fba9-45cd-860a-6567e43f7688 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:39.516581 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 466/917] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 14 20:21:39 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:21:39.538731 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 452/918] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 14 20:21:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:21:40.016972 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99d067-4dae-4423-9b2e-92dd118b5d92 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:40.017368 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99d067-4dae-4423-9b2e-92dd118b5d92 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:40.017574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99d067-4dae-4423-9b2e-92dd118b5d92 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:40.019423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 467/919] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:40.024693 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98628283-4159-4c2b-8434-7cee68fdd30b None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:40.024985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98628283-4159-4c2b-8434-7cee68fdd30b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:40.025269 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98628283-4159-4c2b-8434-7cee68fdd30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:40.043206 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-98628283-4159-4c2b-8434-7cee68fdd30b None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:40.044998 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-98628283-4159-4c2b-8434-7cee68fdd30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:40.121536 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 453/920] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:40.143762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99d067-4dae-4423-9b2e-92dd118b5d92 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:40.257693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99d067-4dae-4423-9b2e-92dd118b5d92 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:40.295927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20b12e-5f2f-4544-919b-f0f823e5dd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:40.296747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20b12e-5f2f-4544-919b-f0f823e5dd86 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:40.297038 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20b12e-5f2f-4544-919b-f0f823e5dd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:40.297379 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20b12e-5f2f-4544-919b-f0f823e5dd86 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:40.298052 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20b12e-5f2f-4544-919b-f0f823e5dd86 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:40.299536 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20b12e-5f2f-4544-919b-f0f823e5dd86 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:40.312823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-2f20b12e-5f2f-4544-919b-f0f823e5dd86 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 20:21:40.312823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:40.312823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:40.312823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:40.312823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:40.312823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:40.312823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:40.312823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:40.312823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 20:21:40.312823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:40.312823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:40.312823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:40.312823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:40.312823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:40.312823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:40.312823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:40.313807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:40.313807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:40.313807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:40.313807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:40.313807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:40.313807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:40.313807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:40.313807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:40.313807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:40.313807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:40.313807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:40.313807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:40.313807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:40.313807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:40.313807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:40.313807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:40.314626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:40.314626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:40.314626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:40.314626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:40.314626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:40.314626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:40.314626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:40.314626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:40.314626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:40.314626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:40.314626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:40.314626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:40.314626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:40.314626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 20:21:40.314626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:40.314626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 468/921] 10.222.0.52 () {56 vars in 1132 bytes} [Thu Jul 14 20:21:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:40.321583 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98628283-4159-4c2b-8434-7cee68fdd30b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:40.352740 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:40.436213 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98628283-4159-4c2b-8434-7cee68fdd30b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:40.475890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e216a61-b504-4b5f-8cf5-8b4d4ea12453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:40.476537 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e216a61-b504-4b5f-8cf5-8b4d4ea12453 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:40.476829 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e216a61-b504-4b5f-8cf5-8b4d4ea12453 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:40.477111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e216a61-b504-4b5f-8cf5-8b4d4ea12453 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:40.477822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e216a61-b504-4b5f-8cf5-8b4d4ea12453 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:40.478130 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e216a61-b504-4b5f-8cf5-8b4d4ea12453 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:40.479538 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e216a61-b504-4b5f-8cf5-8b4d4ea12453 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:40.484859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 454/922] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 14 20:21:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:40.490988 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20b12e-5f2f-4544-919b-f0f823e5dd86 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:40.504172 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:40.603463 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20b12e-5f2f-4544-919b-f0f823e5dd86 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:40.643027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233d5c5-8dd8-4a21-86f3-cbf8af405e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:40.643027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233d5c5-8dd8-4a21-86f3-cbf8af405e65 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:40.643027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233d5c5-8dd8-4a21-86f3-cbf8af405e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:40.643027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233d5c5-8dd8-4a21-86f3-cbf8af405e65 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:40.643386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233d5c5-8dd8-4a21-86f3-cbf8af405e65 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:40.643386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233d5c5-8dd8-4a21-86f3-cbf8af405e65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:40.644865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233d5c5-8dd8-4a21-86f3-cbf8af405e65 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:40.654971 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 469/923] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 14 20:21:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:40.660019 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e216a61-b504-4b5f-8cf5-8b4d4ea12453 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:40.774285 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:40.790962 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e216a61-b504-4b5f-8cf5-8b4d4ea12453 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:40.829233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d72533-d969-4a0f-b15c-58e7419aa858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:40.830274 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d72533-d969-4a0f-b15c-58e7419aa858 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:40.830786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d72533-d969-4a0f-b15c-58e7419aa858 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:40.831296 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d72533-d969-4a0f-b15c-58e7419aa858 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:40.832242 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d72533-d969-4a0f-b15c-58e7419aa858 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:40.833832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d72533-d969-4a0f-b15c-58e7419aa858 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:40.863198 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 455/924] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 14 20:21:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:21:41.337803 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec22ea-23ed-453b-a9c6-00b150ecba4c None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:41.338205 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec22ea-23ed-453b-a9c6-00b150ecba4c None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:41.338389 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec22ea-23ed-453b-a9c6-00b150ecba4c None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:41.340046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 470/925] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:41.346195 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e618f11-ce4f-4cec-be42-37b25612e437 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:41.346804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e618f11-ce4f-4cec-be42-37b25612e437 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:41.347332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e618f11-ce4f-4cec-be42-37b25612e437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:41.365534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-7e618f11-ce4f-4cec-be42-37b25612e437 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:41.367097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7e618f11-ce4f-4cec-be42-37b25612e437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:41.446404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 456/926] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:41.468487 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec22ea-23ed-453b-a9c6-00b150ecba4c None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:41.548166 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:41.582581 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec22ea-23ed-453b-a9c6-00b150ecba4c None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:41.620850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2d391-50d9-4f80-b2d8-61ffc1cf8c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:41.622085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2d391-50d9-4f80-b2d8-61ffc1cf8c6a None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:41.622624 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2d391-50d9-4f80-b2d8-61ffc1cf8c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:41.623160 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2d391-50d9-4f80-b2d8-61ffc1cf8c6a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:41.636756 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2d391-50d9-4f80-b2d8-61ffc1cf8c6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:41.645672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185c982-d74c-435b-a64b-d560c1b0a4c9 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:41.646256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185c982-d74c-435b-a64b-d560c1b0a4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:41.646393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-dff2d391-50d9-4f80-b2d8-61ffc1cf8c6a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 20:21:41.646393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:41.646393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:41.646393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:41.646393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:41.646393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:41.646393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:41.646393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:41.646393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:41.646393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:41.646393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:41.646393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:41.646393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:41.646393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:41.646393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:41.646393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:41.648175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:41.648175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:41.648175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:41.648175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:41.648175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:41.648175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:41.648175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:41.648175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:41.648175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:41.648175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:41.648175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:41.648175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:41.648175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:41.648175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:41.648175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:41.648175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:41.649859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:41.649859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:41.649859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:41.649859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:41.649859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:41.649859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:41.649859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:41.649859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:41.649859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:41.649859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:41.649859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 20:21:41.649859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:41.651172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185c982-d74c-435b-a64b-d560c1b0a4c9 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:41.651327 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 471/927] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jul 14 20:21:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:41.651453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 457/928] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:21:41.654768 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a1206-3bef-4438-8f15-53f63d86edbc None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:41.654966 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185c982-d74c-435b-a64b-d560c1b0a4c9 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:41.655071 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a1206-3bef-4438-8f15-53f63d86edbc None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:41.655350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a1206-3bef-4438-8f15-53f63d86edbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:41.673348 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-d54a1206-3bef-4438-8f15-53f63d86edbc None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:41.674982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d54a1206-3bef-4438-8f15-53f63d86edbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:41.746364 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:41.755086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 472/929] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:41.755086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad375f-c533-4de1-927d-b61bad5faa69 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:41.755086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad375f-c533-4de1-927d-b61bad5faa69 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:41.755086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad375f-c533-4de1-927d-b61bad5faa69 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:41.755086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 473/930] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:41.758585 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df8078-abde-43be-87a5-e70ada86a034 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:41.759075 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df8078-abde-43be-87a5-e70ada86a034 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:41.759506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df8078-abde-43be-87a5-e70ada86a034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:41.780121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-28df8078-abde-43be-87a5-e70ada86a034 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:41.781306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-28df8078-abde-43be-87a5-e70ada86a034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:41.796381 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185c982-d74c-435b-a64b-d560c1b0a4c9 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:41.834507 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a1299-a737-4373-8f0d-786260e27200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:41.835220 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a1299-a737-4373-8f0d-786260e27200 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:41.835506 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a1299-a737-4373-8f0d-786260e27200 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:41.835787 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a1299-a737-4373-8f0d-786260e27200 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:41.836547 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a1299-a737-4373-8f0d-786260e27200 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:41.836858 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a1299-a737-4373-8f0d-786260e27200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:41.838817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a1299-a737-4373-8f0d-786260e27200 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:41.843589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:41.854984 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 458/931] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 14 20:21:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:41.854984 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a1299-a737-4373-8f0d-786260e27200 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:41.861660 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 474/932] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:41.862704 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df8078-abde-43be-87a5-e70ada86a034 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:41.978663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df8078-abde-43be-87a5-e70ada86a034 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:41.986674 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a1299-a737-4373-8f0d-786260e27200 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:42.018220 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:42.023324 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d982102-bd57-441d-9102-b0b8bd4213a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:42.024248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d982102-bd57-441d-9102-b0b8bd4213a8 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:42.024533 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d982102-bd57-441d-9102-b0b8bd4213a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:42.024808 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d982102-bd57-441d-9102-b0b8bd4213a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:42.024988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34a8ec-0fd9-4cc7-9868-52ea245b1dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:42.025872 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34a8ec-0fd9-4cc7-9868-52ea245b1dec None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:42.026164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34a8ec-0fd9-4cc7-9868-52ea245b1dec None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:42.026443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34a8ec-0fd9-4cc7-9868-52ea245b1dec None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:42.027389 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34a8ec-0fd9-4cc7-9868-52ea245b1dec None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:42.029861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34a8ec-0fd9-4cc7-9868-52ea245b1dec None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:42.036817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d982102-bd57-441d-9102-b0b8bd4213a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:42.038292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d982102-bd57-441d-9102-b0b8bd4213a8 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:42.044170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-2c34a8ec-0fd9-4cc7-9868-52ea245b1dec None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 20:21:42.044170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:42.044170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:42.044170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:42.044170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:42.044170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:42.044170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:42.044170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:42.044170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 20:21:42.044170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:42.044170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:42.044170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:42.044170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:42.044170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:42.044170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:42.044170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:42.044170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:42.045320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:42.045320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:42.045320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:42.045320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:42.045320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:42.045320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:42.045320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:42.045320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:42.045320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:42.045320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:42.045320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:42.045320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:42.045320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:42.045320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:42.045320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:42.045320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:42.046172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:42.046172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:42.046172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:42.046172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:42.046172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:42.046172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:42.046172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:42.046172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:42.046172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:42.046172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:42.046172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:42.046172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:42.046172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 20:21:42.046172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:42.046172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 459/933] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 14 20:21:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:42.046988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34a8ec-0fd9-4cc7-9868-52ea245b1dec None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:42.063265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-0d982102-bd57-441d-9102-b0b8bd4213a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:42.063265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:42.063265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:42.063265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:42.063265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:42.063265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:42.063265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:42.063265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:42.063265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:42.063265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:42.063265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:42.063265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:42.063265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:42.063265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:42.063265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:42.064237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:42.064237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:42.064237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:42.064237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:42.064237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:42.064237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:42.064237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:42.064237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:42.064237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:42.064237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:42.064237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:42.064237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:42.064237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:42.064237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:42.064237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:42.064237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:42.064237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:42.065145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:42.065145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:42.065145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:42.065145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:42.065145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 475/934] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:42.066111 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d982102-bd57-441d-9102-b0b8bd4213a8 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:42.116551 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:42.160090 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34a8ec-0fd9-4cc7-9868-52ea245b1dec None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:42.180362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d982102-bd57-441d-9102-b0b8bd4213a8 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:42.196089 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9ad55-f2f2-4f97-b653-13725432f9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:42.201515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9ad55-f2f2-4f97-b653-13725432f9f9 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:42.201515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9ad55-f2f2-4f97-b653-13725432f9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:42.201515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9ad55-f2f2-4f97-b653-13725432f9f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:42.201515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9ad55-f2f2-4f97-b653-13725432f9f9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:42.201515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9ad55-f2f2-4f97-b653-13725432f9f9 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:42.202232 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: WARNING keystone.server.flask.common [None req-c9b9ad55-f2f2-4f97-b653-13725432f9f9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 14 20:21:42.219800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.sql.core [None req-c9b9ad55-f2f2-4f97-b653-13725432f9f9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 20:21:42.219800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 20:21:42.219800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [parameters: {'id': '9a1f1699a28f49739ede7921b8f0fb3d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 20:21:42.219800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80177) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 20:21:42.223655 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d44c2f-d9e6-48a4-bea4-fd57e0709c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:42.224332 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d44c2f-d9e6-48a4-bea4-fd57e0709c03 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:42.224630 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d44c2f-d9e6-48a4-bea4-fd57e0709c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:42.224942 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d44c2f-d9e6-48a4-bea4-fd57e0709c03 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:42.225714 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d44c2f-d9e6-48a4-bea4-fd57e0709c03 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:42.226015 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d44c2f-d9e6-48a4-bea4-fd57e0709c03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:42.227153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-c9b9ad55-f2f2-4f97-b653-13725432f9f9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9a1f1699a28f49739ede7921b8f0fb3d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9a1f1699a28f49739ede7921b8f0fb3d. Jul 14 20:21:42.227153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:42.227153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 20:21:42.227153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 20:21:42.227153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 20:21:42.227153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 20:21:42.227153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 20:21:42.227153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 20:21:42.227153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 20:21:42.227153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application conn.query(q) Jul 14 20:21:42.227153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 20:21:42.227153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 20:21:42.227153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 20:21:42.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application result.read() Jul 14 20:21:42.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 20:21:42.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 20:21:42.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 20:21:42.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 20:21:42.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 20:21:42.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 20:21:42.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 20:21:42.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 20:21:42.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 20:21:42.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:42.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 20:21:42.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:42.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:42.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 20:21:42.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 20:21:42.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 14 20:21:42.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 14 20:21:42.228950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 20:21:42.228950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application next(self.gen) Jul 14 20:21:42.228950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 20:21:42.228950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application yield resource Jul 14 20:21:42.228950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 20:21:42.228950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application next(self.gen) Jul 14 20:21:42.228950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 20:21:42.228950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 20:21:42.228950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 20:21:42.228950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 20:21:42.228950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 20:21:42.228950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise self.value Jul 14 20:21:42.228950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 20:21:42.228950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 20:21:42.228950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 20:21:42.228950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application session.commit() Jul 14 20:21:42.228950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 20:21:42.229859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 20:21:42.229859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 20:21:42.229859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 20:21:42.229859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 20:21:42.229859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self.session.flush() Jul 14 20:21:42.229859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 20:21:42.229859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 20:21:42.229859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 20:21:42.229859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 20:21:42.229859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 20:21:42.229859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application compat.raise_( Jul 14 20:21:42.229859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 20:21:42.229859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception Jul 14 20:21:42.229859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 20:21:42.229859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 20:21:42.229859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 20:21:42.229859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 20:21:42.230782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 20:21:42.230782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 20:21:42.230782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 20:21:42.230782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 20:21:42.230782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 20:21:42.230782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 20:21:42.230782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 20:21:42.230782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 20:21:42.230782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 20:21:42.230782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 20:21:42.230782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 20:21:42.230782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 20:21:42.230782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 20:21:42.230782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 20:21:42.230782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 20:21:42.233793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 20:21:42.233793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 20:21:42.233793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception Jul 14 20:21:42.233793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 20:21:42.233793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 20:21:42.233793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 20:21:42.233793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 20:21:42.233793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 20:21:42.233793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 20:21:42.233793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 20:21:42.233793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application conn.query(q) Jul 14 20:21:42.233793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 20:21:42.233793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 20:21:42.233793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 20:21:42.233793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application result.read() Jul 14 20:21:42.233793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 20:21:42.233793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 20:21:42.236207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 20:21:42.236207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 20:21:42.236207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 20:21:42.236207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 20:21:42.236207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 20:21:42.236207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 20:21:42.236207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 20:21:42.236207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 20:21:42.236207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [parameters: {'id': '9a1f1699a28f49739ede7921b8f0fb3d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 20:21:42.236207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 20:21:42.236207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:42.236207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:42.236207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:42.236207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:42.236207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 14 20:21:42.245148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 14 20:21:42.245148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 20:21:42.245148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 20:21:42.245148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 14 20:21:42.245148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:42.245148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:42.245148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:42.245148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:42.245148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:42.245148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:42.245148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:42.245148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:42.245148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:42.245148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:42.245148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:42.245148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:42.245148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:42.248404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:42.248404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 14 20:21:42.248404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 14 20:21:42.248404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:42.248404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:42.248404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 14 20:21:42.248404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 14 20:21:42.248404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9a1f1699a28f49739ede7921b8f0fb3d. Jul 14 20:21:42.248404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:42.250259 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d44c2f-d9e6-48a4-bea4-fd57e0709c03 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:42.250414 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 460/935] 10.222.0.52 () {60 vars in 1140 bytes} [Thu Jul 14 20:21:42 2022] POST /identity/v3/projects => generated 262 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:42.250414 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9ad55-f2f2-4f97-b653-13725432f9f9 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:42.250864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 476/936] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 14 20:21:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:42.250864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d44c2f-d9e6-48a4-bea4-fd57e0709c03 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:42.288860 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:42.347740 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9ad55-f2f2-4f97-b653-13725432f9f9 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:42.354797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d44c2f-d9e6-48a4-bea4-fd57e0709c03 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:42.364915 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:42.385337 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cfb18-074a-42e3-a987-99137bd08717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:42.386058 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cfb18-074a-42e3-a987-99137bd08717 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:42.386290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cfb18-074a-42e3-a987-99137bd08717 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:42.386564 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cfb18-074a-42e3-a987-99137bd08717 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:42.387886 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02cfb18-074a-42e3-a987-99137bd08717 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:42.388193 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02cfb18-074a-42e3-a987-99137bd08717 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:42.389810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02cfb18-074a-42e3-a987-99137bd08717 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:42.395153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac07aab-82a8-4c35-a9ca-690412541d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:42.395956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac07aab-82a8-4c35-a9ca-690412541d29 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:42.396240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac07aab-82a8-4c35-a9ca-690412541d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:42.396516 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac07aab-82a8-4c35-a9ca-690412541d29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:42.404669 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac07aab-82a8-4c35-a9ca-690412541d29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:42.406097 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac07aab-82a8-4c35-a9ca-690412541d29 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:42.412088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 461/937] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:42.412088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cfb18-074a-42e3-a987-99137bd08717 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:42.427613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-fac07aab-82a8-4c35-a9ca-690412541d29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:42.427613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:42.427613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:42.427613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:42.427613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:42.427613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:42.427613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:42.427613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:42.427613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:42.427613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:42.427613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:42.427613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:42.427613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:42.427613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:42.427613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:42.428578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:42.428578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:42.428578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:42.428578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:42.428578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:42.428578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:42.428578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:42.428578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:42.428578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:42.428578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:42.428578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:42.428578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:42.428578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:42.428578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:42.428578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:42.428578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:42.428578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:42.429506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:42.429506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:42.429506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:42.429506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:42.429506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:42.429506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:42.429506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:42.429506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:42.429506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 477/938] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:42.430015 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac07aab-82a8-4c35-a9ca-690412541d29 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:42.526096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cfb18-074a-42e3-a987-99137bd08717 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:42.540540 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:42.549964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac07aab-82a8-4c35-a9ca-690412541d29 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:42.565155 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baef372-ff94-4507-b7b9-098d4995845c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:42.565831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baef372-ff94-4507-b7b9-098d4995845c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:42.566121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baef372-ff94-4507-b7b9-098d4995845c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:42.566399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baef372-ff94-4507-b7b9-098d4995845c None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:42.567146 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baef372-ff94-4507-b7b9-098d4995845c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:42.567458 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baef372-ff94-4507-b7b9-098d4995845c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:42.568836 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baef372-ff94-4507-b7b9-098d4995845c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:42.579181 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 462/939] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 14 20:21:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:42.579998 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baef372-ff94-4507-b7b9-098d4995845c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:42.586094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20497eb-c288-4bab-8b26-d98bd7b30ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:42.586917 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20497eb-c288-4bab-8b26-d98bd7b30ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:42.587294 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20497eb-c288-4bab-8b26-d98bd7b30ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:42.587576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20497eb-c288-4bab-8b26-d98bd7b30ee6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:42.595779 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20497eb-c288-4bab-8b26-d98bd7b30ee6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:42.598037 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20497eb-c288-4bab-8b26-d98bd7b30ee6 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:42.619352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-d20497eb-c288-4bab-8b26-d98bd7b30ee6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 20:21:42.619352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:42.619352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:42.619352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:42.619352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:42.619352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:42.619352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:42.619352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:42.619352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:42.619352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:42.619352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:42.619352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:42.619352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:42.619352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:42.619352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:42.619352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:42.620609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:42.620609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:42.620609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:42.620609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:42.620609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:42.620609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:42.620609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:42.620609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:42.620609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:42.620609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:42.620609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:42.620609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:42.620609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:42.620609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:42.620609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:42.620609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:42.620609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:42.620609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:42.622563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:42.622563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:42.622563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:42.622563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:42.622563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:42.622563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:42.622563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:42.622563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:42.622563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 20:21:42.622563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:42.622563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 478/940] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 14 20:21:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:42.623235 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20497eb-c288-4bab-8b26-d98bd7b30ee6 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:42.660831 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:42.704183 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baef372-ff94-4507-b7b9-098d4995845c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:42.741538 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20497eb-c288-4bab-8b26-d98bd7b30ee6 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:42.749570 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0141e-3518-48e0-a612-6c1242474ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:42.751528 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0141e-3518-48e0-a612-6c1242474ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:42.751858 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0141e-3518-48e0-a612-6c1242474ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:42.752125 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0141e-3518-48e0-a612-6c1242474ca9 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:42.753583 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0141e-3518-48e0-a612-6c1242474ca9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:42.754498 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0141e-3518-48e0-a612-6c1242474ca9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:42.756991 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0141e-3518-48e0-a612-6c1242474ca9 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:42.795047 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 463/941] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 14 20:21:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:42.795047 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0141e-3518-48e0-a612-6c1242474ca9 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:42.796630 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ea151-7f42-42ca-96cc-d3dc918527b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:42.797392 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ea151-7f42-42ca-96cc-d3dc918527b4 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:42.797714 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ea151-7f42-42ca-96cc-d3dc918527b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:42.798007 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ea151-7f42-42ca-96cc-d3dc918527b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:42.798982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ea151-7f42-42ca-96cc-d3dc918527b4 None admin] RBAC: Adding query filter params (interface=public, service_id=9f7c0bba4a7743b593449bb3e84430bd, region_id=RegionOne) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:42.799307 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ea151-7f42-42ca-96cc-d3dc918527b4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:42.800738 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ea151-7f42-42ca-96cc-d3dc918527b4 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:42.808428 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 479/942] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 14 20:21:42 2022] GET /identity/v3/endpoints?service_id=9f7c0bba4a7743b593449bb3e84430bd&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:42.815626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ea151-7f42-42ca-96cc-d3dc918527b4 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:42.848053 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:42.956555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:42.959096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0141e-3518-48e0-a612-6c1242474ca9 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:42.972878 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ea151-7f42-42ca-96cc-d3dc918527b4 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:43.015268 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db96f1b-9548-4b3d-ac76-a01bc5c64860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:43.015970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db96f1b-9548-4b3d-ac76-a01bc5c64860 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:43.016256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db96f1b-9548-4b3d-ac76-a01bc5c64860 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:43.016539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db96f1b-9548-4b3d-ac76-a01bc5c64860 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:43.018457 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db96f1b-9548-4b3d-ac76-a01bc5c64860 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:43.018457 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db96f1b-9548-4b3d-ac76-a01bc5c64860 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:43.018877 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf06df-e010-48ae-8d7b-8cf8ab2e9df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:43.019438 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db96f1b-9548-4b3d-ac76-a01bc5c64860 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:43.020505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf06df-e010-48ae-8d7b-8cf8ab2e9df7 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:43.020802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf06df-e010-48ae-8d7b-8cf8ab2e9df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:43.021085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf06df-e010-48ae-8d7b-8cf8ab2e9df7 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:43.022129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf06df-e010-48ae-8d7b-8cf8ab2e9df7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:43.024316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf06df-e010-48ae-8d7b-8cf8ab2e9df7 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:43.036089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 480/943] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 14 20:21:42 2022] GET /identity/v3/services => generated 1497 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:21:43.036089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fba80a-0fc9-402e-a963-d69c095de13b None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:43.036089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fba80a-0fc9-402e-a963-d69c095de13b None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:43.036405 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fba80a-0fc9-402e-a963-d69c095de13b None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:43.037741 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 481/944] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:43.046117 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd128e-2b97-4278-9071-a1728ae09699 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:43.046117 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd128e-2b97-4278-9071-a1728ae09699 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:43.046117 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd128e-2b97-4278-9071-a1728ae09699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:43.047704 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 464/945] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jul 14 20:21:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:43.053121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db96f1b-9548-4b3d-ac76-a01bc5c64860 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:43.066429 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-90bd128e-2b97-4278-9071-a1728ae09699 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:43.067851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-90bd128e-2b97-4278-9071-a1728ae09699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:43.153940 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 482/946] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 20:21:43.178442 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd128e-2b97-4278-9071-a1728ae09699 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:43.193490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:43.203400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db96f1b-9548-4b3d-ac76-a01bc5c64860 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:43.250500 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912f09f-d2db-4ba4-aba6-eee8e30b88c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:43.251357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912f09f-d2db-4ba4-aba6-eee8e30b88c0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:43.251638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912f09f-d2db-4ba4-aba6-eee8e30b88c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:43.251941 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912f09f-d2db-4ba4-aba6-eee8e30b88c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:43.266440 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912f09f-d2db-4ba4-aba6-eee8e30b88c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:43.267931 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912f09f-d2db-4ba4-aba6-eee8e30b88c0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:43.287183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:43.289218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-b912f09f-d2db-4ba4-aba6-eee8e30b88c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:43.289218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:43.289218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:43.289218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:43.289218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:43.289218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:43.289218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:43.289218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:43.289218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:43.289218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:43.289218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:43.289218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:43.289218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:43.289218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:43.289218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:43.290110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:43.290110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:43.290110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:43.290110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:43.290110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:43.290110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:43.290110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:43.290110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:43.290110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:43.290110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:43.290110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:43.290110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:43.290110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:43.290110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:43.290110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:43.290110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:43.290110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:43.291597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:43.291597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:43.291597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:43.291597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:43.291597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 465/947] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:43.297105 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912f09f-d2db-4ba4-aba6-eee8e30b88c0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:43.305297 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd128e-2b97-4278-9071-a1728ae09699 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:43.346185 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459da37a-0ad3-450e-8287-d59c62b80eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:43.347127 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459da37a-0ad3-450e-8287-d59c62b80eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:43.347484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459da37a-0ad3-450e-8287-d59c62b80eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:43.347833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459da37a-0ad3-450e-8287-d59c62b80eb8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:43.348677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459da37a-0ad3-450e-8287-d59c62b80eb8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:43.350138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459da37a-0ad3-450e-8287-d59c62b80eb8 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:43.362679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-459da37a-0ad3-450e-8287-d59c62b80eb8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 20:21:43.362679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:43.362679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:43.362679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:43.362679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:43.362679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:43.362679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:43.362679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:43.362679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 20:21:43.362679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:43.362679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:43.362679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:43.362679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:43.362679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:43.362679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:43.362679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:43.362679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:43.363983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:43.363983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:43.363983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:43.363983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:43.363983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:43.363983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:43.363983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:43.363983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:43.363983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:43.363983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:43.363983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:43.363983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:43.363983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:43.363983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:43.363983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:43.363983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:43.365030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:43.365030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:43.365030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:43.365030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:43.365030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:43.365030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:43.365030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:43.365030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:43.365030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:43.365030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:43.365030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:43.365030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:43.365030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 20:21:43.365030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:43.372134 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 483/948] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:43.373154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459da37a-0ad3-450e-8287-d59c62b80eb8 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:43.422418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912f09f-d2db-4ba4-aba6-eee8e30b88c0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:43.473278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69601c-f87b-4c54-9ce5-5fed073846f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:43.474072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69601c-f87b-4c54-9ce5-5fed073846f5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:43.474424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69601c-f87b-4c54-9ce5-5fed073846f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:43.474770 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69601c-f87b-4c54-9ce5-5fed073846f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:43.475595 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69601c-f87b-4c54-9ce5-5fed073846f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:43.475978 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69601c-f87b-4c54-9ce5-5fed073846f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:43.477428 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69601c-f87b-4c54-9ce5-5fed073846f5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:43.495765 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 466/949] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:43.497428 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69601c-f87b-4c54-9ce5-5fed073846f5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:43.512797 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:43.520312 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459da37a-0ad3-450e-8287-d59c62b80eb8 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:43.568432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024ff33-5958-48d3-bd3d-19acb61dde87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:43.570402 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024ff33-5958-48d3-bd3d-19acb61dde87 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:43.570852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024ff33-5958-48d3-bd3d-19acb61dde87 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:43.571319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024ff33-5958-48d3-bd3d-19acb61dde87 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:43.572366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024ff33-5958-48d3-bd3d-19acb61dde87 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:43.573152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024ff33-5958-48d3-bd3d-19acb61dde87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:43.575084 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024ff33-5958-48d3-bd3d-19acb61dde87 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:43.582406 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 484/950] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 14 20:21:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:43.588589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024ff33-5958-48d3-bd3d-19acb61dde87 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:43.609605 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:43.640537 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69601c-f87b-4c54-9ce5-5fed073846f5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:43.682819 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42298af2-15a7-46c9-8676-58f407784712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:43.684203 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42298af2-15a7-46c9-8676-58f407784712 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:43.684749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42298af2-15a7-46c9-8676-58f407784712 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:43.685217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42298af2-15a7-46c9-8676-58f407784712 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:43.693224 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42298af2-15a7-46c9-8676-58f407784712 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:43.695145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42298af2-15a7-46c9-8676-58f407784712 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:43.711473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-42298af2-15a7-46c9-8676-58f407784712 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:43.711473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:43.711473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:43.711473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:43.711473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:43.711473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:43.711473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:43.711473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:43.711473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:43.711473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:43.711473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:43.711473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:43.711473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:43.711473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:43.711473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:43.712875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:43.712875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:43.712875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:43.712875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:43.712875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:43.712875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:43.712875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:43.712875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:43.712875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:43.712875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:43.712875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:43.712875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:43.712875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:43.712875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:43.712875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:43.712875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:43.712875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:43.713766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:43.713766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:43.713766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:43.713766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:43.713766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:43.713766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:43.713766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:43.713766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:43.716921 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 467/951] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:43.722418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42298af2-15a7-46c9-8676-58f407784712 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:43.736754 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024ff33-5958-48d3-bd3d-19acb61dde87 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:43.776537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d3b94-8667-4bdb-b366-be732ce6baab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:43.777461 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d3b94-8667-4bdb-b366-be732ce6baab None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:43.778290 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d3b94-8667-4bdb-b366-be732ce6baab None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:43.778568 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d3b94-8667-4bdb-b366-be732ce6baab None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:43.779313 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d3b94-8667-4bdb-b366-be732ce6baab None admin] RBAC: Adding query filter params (type=compute) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:43.779624 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d3b94-8667-4bdb-b366-be732ce6baab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:43.781742 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d3b94-8667-4bdb-b366-be732ce6baab None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:43.792581 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 485/952] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 14 20:21:43 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:43.803184 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:43.844122 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42298af2-15a7-46c9-8676-58f407784712 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:43.883364 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d402fe-7ffb-4d9d-a8dc-4aa1275d31d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:43.884046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d402fe-7ffb-4d9d-a8dc-4aa1275d31d2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:43.884333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d402fe-7ffb-4d9d-a8dc-4aa1275d31d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:43.884606 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d402fe-7ffb-4d9d-a8dc-4aa1275d31d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:43.885418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d402fe-7ffb-4d9d-a8dc-4aa1275d31d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:43.885725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d402fe-7ffb-4d9d-a8dc-4aa1275d31d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:43.887239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d402fe-7ffb-4d9d-a8dc-4aa1275d31d2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:43.901708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 468/953] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:43.908306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d3b94-8667-4bdb-b366-be732ce6baab None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:44.002764 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:44.026056 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d3b94-8667-4bdb-b366-be732ce6baab None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:44.062069 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f487610-be20-4b3d-be2e-e8029e55a8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:44.062994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f487610-be20-4b3d-be2e-e8029e55a8eb None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:44.063284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f487610-be20-4b3d-be2e-e8029e55a8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:44.063563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f487610-be20-4b3d-be2e-e8029e55a8eb None admin] PATH_INFO: `/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/4f75b8b484384554ad59aa20535875e9/roles/f932ef767ec247d8ad3823e16f1b0306` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:44.065809 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f487610-be20-4b3d-be2e-e8029e55a8eb None admin] RBAC: Authorizing `identity:create_grant(project_id=f33c8ac5154f46a89d1a0dc11fb58cf4, user_id=4f75b8b484384554ad59aa20535875e9, role_id=f932ef767ec247d8ad3823e16f1b0306)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:44.067269 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f487610-be20-4b3d-be2e-e8029e55a8eb None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:44.083470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 486/954] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:21:43 2022] PUT /identity/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/4f75b8b484384554ad59aa20535875e9/roles/f932ef767ec247d8ad3823e16f1b0306 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:21:45.226455 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b1959-1212-46b0-b5cb-3b743569dbb5 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:45.226996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b1959-1212-46b0-b5cb-3b743569dbb5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:45.227102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b1959-1212-46b0-b5cb-3b743569dbb5 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:45.228548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 469/955] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:45.234077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57a423-f588-4e84-b532-36bbd442fe75 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:45.234524 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57a423-f588-4e84-b532-36bbd442fe75 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:45.234621 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57a423-f588-4e84-b532-36bbd442fe75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:45.253384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-7a57a423-f588-4e84-b532-36bbd442fe75 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:45.254669 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-7a57a423-f588-4e84-b532-36bbd442fe75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:45.328379 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 487/956] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:45.354292 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b1959-1212-46b0-b5cb-3b743569dbb5 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:45.424568 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e22d63-ddaf-4321-a730-618d6df4f958 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:45.424861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e22d63-ddaf-4321-a730-618d6df4f958 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:45.425141 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e22d63-ddaf-4321-a730-618d6df4f958 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:45.426498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 488/957] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:21:45.432017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f28454-7acc-4820-a7ba-ae42c44f4c54 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:45.432303 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f28454-7acc-4820-a7ba-ae42c44f4c54 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:45.432583 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f28454-7acc-4820-a7ba-ae42c44f4c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:45.451697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-29f28454-7acc-4820-a7ba-ae42c44f4c54 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:45.452897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-29f28454-7acc-4820-a7ba-ae42c44f4c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:45.469376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b1959-1212-46b0-b5cb-3b743569dbb5 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:45.498994 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:45.510685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f8c71-d5f8-47e8-ba5c-1bcc508484ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:45.511532 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f8c71-d5f8-47e8-ba5c-1bcc508484ac None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:45.511817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f8c71-d5f8-47e8-ba5c-1bcc508484ac None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:45.512634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f8c71-d5f8-47e8-ba5c-1bcc508484ac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:45.513640 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f8c71-d5f8-47e8-ba5c-1bcc508484ac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:45.514782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f8c71-d5f8-47e8-ba5c-1bcc508484ac None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:45.528992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-b39f8c71-d5f8-47e8-ba5c-1bcc508484ac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 20:21:45.528992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:45.528992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:45.528992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:45.528992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:45.528992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:45.528992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:45.528992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:45.528992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 20:21:45.528992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:45.528992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:45.528992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:45.528992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:45.528992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:45.528992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:45.528992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:45.528992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:45.530160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:45.530160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:45.530160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:45.530160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:45.530160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:45.530160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:45.530160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:45.530160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:45.530160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:45.530160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:45.530160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:45.530160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:45.530160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:45.530160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:45.530160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:45.530160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:45.531689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:45.531689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:45.531689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:45.531689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:45.531689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:45.531689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:45.531689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:45.531689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:45.531689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:45.531689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:45.531689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:45.531689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:45.531689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 20:21:45.531689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:45.531689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 470/958] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 14 20:21:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:45.536543 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f8c71-d5f8-47e8-ba5c-1bcc508484ac None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:45.544672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:45.558756 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 489/959] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 20:21:45.578245 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f28454-7acc-4820-a7ba-ae42c44f4c54 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:45.677978 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f8c71-d5f8-47e8-ba5c-1bcc508484ac None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:45.697523 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f28454-7acc-4820-a7ba-ae42c44f4c54 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:45.723584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923057ce-53ed-4dfc-8ece-836a096be9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:45.724917 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923057ce-53ed-4dfc-8ece-836a096be9db None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:45.725464 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923057ce-53ed-4dfc-8ece-836a096be9db None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:45.725942 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923057ce-53ed-4dfc-8ece-836a096be9db None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:45.726862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923057ce-53ed-4dfc-8ece-836a096be9db None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:45.727397 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923057ce-53ed-4dfc-8ece-836a096be9db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:45.729086 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923057ce-53ed-4dfc-8ece-836a096be9db None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:45.740437 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 471/960] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 14 20:21:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:45.741713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923057ce-53ed-4dfc-8ece-836a096be9db None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:45.743489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b3ce0-2369-42be-b23e-a6295b9a9045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:45.744171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b3ce0-2369-42be-b23e-a6295b9a9045 None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:45.744458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b3ce0-2369-42be-b23e-a6295b9a9045 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:45.744735 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b3ce0-2369-42be-b23e-a6295b9a9045 None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:45.745486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b3ce0-2369-42be-b23e-a6295b9a9045 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:45.748324 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b3ce0-2369-42be-b23e-a6295b9a9045 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:45.749714 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: WARNING keystone.server.flask.common [None req-097b3ce0-2369-42be-b23e-a6295b9a9045 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 14 20:21:45.767091 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.sql.core [None req-097b3ce0-2369-42be-b23e-a6295b9a9045 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 20:21:45.767091 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 20:21:45.767091 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [parameters: {'user_id': 'f729c90e7fc34723becc2dad16e7450b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 20:21:45.767091 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80178) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 20:21:45.789896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-097b3ce0-2369-42be-b23e-a6295b9a9045 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 20:21:45.789896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:45.789896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 20:21:45.789896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 20:21:45.789896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 20:21:45.789896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 20:21:45.789896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 20:21:45.789896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 20:21:45.789896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 20:21:45.789896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application conn.query(q) Jul 14 20:21:45.789896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 20:21:45.789896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 20:21:45.789896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 20:21:45.789896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application result.read() Jul 14 20:21:45.789896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 20:21:45.790924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 20:21:45.790924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 20:21:45.790924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 20:21:45.790924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 20:21:45.790924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 20:21:45.790924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 20:21:45.790924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 20:21:45.790924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 20:21:45.790924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:45.790924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 20:21:45.790924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:45.790924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:45.790924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 20:21:45.790924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 20:21:45.790924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 14 20:21:45.790924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 14 20:21:45.790924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 20:21:45.790924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application next(self.gen) Jul 14 20:21:45.792252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 20:21:45.792252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application yield resource Jul 14 20:21:45.792252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 20:21:45.792252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application next(self.gen) Jul 14 20:21:45.792252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 20:21:45.792252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 20:21:45.792252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 20:21:45.792252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 20:21:45.792252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 20:21:45.792252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise self.value Jul 14 20:21:45.792252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 20:21:45.792252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 20:21:45.792252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 20:21:45.792252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application session.commit() Jul 14 20:21:45.792252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 20:21:45.792252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 20:21:45.792252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 20:21:45.793143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 20:21:45.793143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 20:21:45.793143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self.session.flush() Jul 14 20:21:45.793143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 20:21:45.793143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 20:21:45.793143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 20:21:45.793143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 20:21:45.793143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 20:21:45.793143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application compat.raise_( Jul 14 20:21:45.793143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 20:21:45.793143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception Jul 14 20:21:45.793143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 20:21:45.793143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 20:21:45.793143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 20:21:45.793143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 20:21:45.793143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 20:21:45.793143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 20:21:45.794212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 20:21:45.794212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 20:21:45.794212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 14 20:21:45.794212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 14 20:21:45.794212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 20:21:45.794212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 20:21:45.794212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 20:21:45.794212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 20:21:45.794212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 20:21:45.794212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 20:21:45.794212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 20:21:45.794212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 20:21:45.794212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 20:21:45.794212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 20:21:45.794212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 20:21:45.794212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception Jul 14 20:21:45.795927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 20:21:45.795927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 20:21:45.795927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 20:21:45.795927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 20:21:45.795927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 20:21:45.795927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 20:21:45.795927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 20:21:45.795927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application conn.query(q) Jul 14 20:21:45.795927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 20:21:45.795927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 20:21:45.795927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 20:21:45.795927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application result.read() Jul 14 20:21:45.795927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 20:21:45.795927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 20:21:45.795927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 20:21:45.795927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 20:21:45.798033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 20:21:45.798033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 20:21:45.798033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 20:21:45.798033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 20:21:45.798033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 20:21:45.798033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 20:21:45.798033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f729c90e7fc34723becc2dad16e7450b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 20:21:45.798033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 20:21:45.798033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:45.798033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:45.798033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:45.798033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:45.798033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:45.798033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:45.798033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:45.800147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:45.800147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:45.800147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:45.800147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:45.800147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:45.800147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:45.800147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:45.800147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 14 20:21:45.800147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 14 20:21:45.800147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:45.800147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:45.800147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:45.800147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:45.800147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:45.800147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:45.800147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 14 20:21:45.800147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 14 20:21:45.802400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 14 20:21:45.802400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 14 20:21:45.802400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 20:21:45.802400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 20:21:45.802400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 20:21:45.802400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:45.802400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 490/961] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jul 14 20:21:45 2022] POST /identity/v3/users => generated 160 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:45.802400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b3ce0-2369-42be-b23e-a6295b9a9045 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:45.825545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:45.878676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923057ce-53ed-4dfc-8ece-836a096be9db None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:45.883061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:45.921585 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281210ed-2344-441b-ab94-e008707e6397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:45.922256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281210ed-2344-441b-ab94-e008707e6397 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:45.922543 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281210ed-2344-441b-ab94-e008707e6397 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:45.922822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281210ed-2344-441b-ab94-e008707e6397 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:45.924026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281210ed-2344-441b-ab94-e008707e6397 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:45.924343 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281210ed-2344-441b-ab94-e008707e6397 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:45.925773 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281210ed-2344-441b-ab94-e008707e6397 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:45.936272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 472/962] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 14 20:21:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:45.936272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e27f5-f22b-482a-a676-f7975f54e765 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:45.936272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e27f5-f22b-482a-a676-f7975f54e765 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:45.936272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e27f5-f22b-482a-a676-f7975f54e765 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:45.936272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 473/963] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:45.939527 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e27f5-f22b-482a-a676-f7975f54e765 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:45.943273 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b3ce0-2369-42be-b23e-a6295b9a9045 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:45.984128 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc48afc-b649-4adb-9ab2-2126da474844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:45.985219 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc48afc-b649-4adb-9ab2-2126da474844 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:45.985543 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc48afc-b649-4adb-9ab2-2126da474844 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:45.985819 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc48afc-b649-4adb-9ab2-2126da474844 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:45.994922 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc48afc-b649-4adb-9ab2-2126da474844 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:45.997261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc48afc-b649-4adb-9ab2-2126da474844 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:46.013158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-6bc48afc-b649-4adb-9ab2-2126da474844 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 20:21:46.013158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:46.013158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:46.013158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:46.013158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:46.013158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:46.013158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:46.013158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:46.013158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:46.013158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:46.013158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:46.013158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:46.013158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:46.013158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:46.013158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:46.013158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:46.014292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:46.014292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:46.014292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:46.014292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:46.014292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:46.014292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:46.014292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:46.014292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:46.014292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:46.014292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:46.014292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:46.014292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:46.014292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:46.014292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:46.014292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:46.014292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:46.014292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:46.014292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:46.015822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:46.015822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:46.015822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:46.015822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:46.015822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:46.015822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:46.015822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:46.015822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:46.015822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 20:21:46.015822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:46.015822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 491/964] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:21:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:46.015822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f469da-692f-4e75-b893-4d49ef7772cd None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:46.016520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f469da-692f-4e75-b893-4d49ef7772cd None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:46.016520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f469da-692f-4e75-b893-4d49ef7772cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:46.035142 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-e4f469da-692f-4e75-b893-4d49ef7772cd None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:46.036915 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e4f469da-692f-4e75-b893-4d49ef7772cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:46.054749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e27f5-f22b-482a-a676-f7975f54e765 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:46.074519 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:46.094213 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf502f1-4c75-45ea-a735-4a1eb97cfe3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:46.094879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf502f1-4c75-45ea-a735-4a1eb97cfe3c None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:46.095198 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf502f1-4c75-45ea-a735-4a1eb97cfe3c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:46.095477 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf502f1-4c75-45ea-a735-4a1eb97cfe3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:46.096232 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf502f1-4c75-45ea-a735-4a1eb97cfe3c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:46.097611 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf502f1-4c75-45ea-a735-4a1eb97cfe3c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:46.126957 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 492/965] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:46.126957 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f469da-692f-4e75-b893-4d49ef7772cd None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:46.130619 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 474/966] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 14 20:21:45 2022] POST /identity/v3/endpoints => generated 349 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:21:46.146047 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf502f1-4c75-45ea-a735-4a1eb97cfe3c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:46.197458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:46.260944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f469da-692f-4e75-b893-4d49ef7772cd None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:46.276025 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf502f1-4c75-45ea-a735-4a1eb97cfe3c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:46.301448 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea014a-6d83-4f4c-acf6-a1e9f302bdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:46.302462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea014a-6d83-4f4c-acf6-a1e9f302bdcb None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:46.303179 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea014a-6d83-4f4c-acf6-a1e9f302bdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:46.303179 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea014a-6d83-4f4c-acf6-a1e9f302bdcb None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:46.304385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea014a-6d83-4f4c-acf6-a1e9f302bdcb None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:46.304385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea014a-6d83-4f4c-acf6-a1e9f302bdcb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:46.306130 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea014a-6d83-4f4c-acf6-a1e9f302bdcb None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:46.323133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb466e-7d57-44c5-babd-4cc5b3c6171f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:46.324001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb466e-7d57-44c5-babd-4cc5b3c6171f None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:46.324761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb466e-7d57-44c5-babd-4cc5b3c6171f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:46.324761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb466e-7d57-44c5-babd-4cc5b3c6171f None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:46.326094 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb466e-7d57-44c5-babd-4cc5b3c6171f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:46.326702 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb466e-7d57-44c5-babd-4cc5b3c6171f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:46.347186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 493/967] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 14 20:21:46 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:46.348556 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782eddf5-2e4b-44c5-ba5d-24a3ca181ded None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:46.348845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782eddf5-2e4b-44c5-ba5d-24a3ca181ded None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:46.349121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782eddf5-2e4b-44c5-ba5d-24a3ca181ded None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:46.350544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 494/968] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:46.356804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16fab1-6ab3-4cad-91af-474891ea8b60 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:46.357128 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16fab1-6ab3-4cad-91af-474891ea8b60 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:46.357386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16fab1-6ab3-4cad-91af-474891ea8b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:46.360318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 475/969] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 14 20:21:46 2022] POST /identity/v3/services => generated 242 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:21:46.383717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-0f16fab1-6ab3-4cad-91af-474891ea8b60 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:46.385535 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-0f16fab1-6ab3-4cad-91af-474891ea8b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:46.487600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 495/970] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 20:21:46.511305 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb466e-7d57-44c5-babd-4cc5b3c6171f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:46.549750 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:46.632772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb466e-7d57-44c5-babd-4cc5b3c6171f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:46.683663 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01dd04-60d9-48fb-ac2f-07aa33ad42f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:46.684488 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01dd04-60d9-48fb-ac2f-07aa33ad42f3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:46.684793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01dd04-60d9-48fb-ac2f-07aa33ad42f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:46.685073 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01dd04-60d9-48fb-ac2f-07aa33ad42f3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:46.697396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01dd04-60d9-48fb-ac2f-07aa33ad42f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:46.714675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-7c01dd04-60d9-48fb-ac2f-07aa33ad42f3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 20:21:46.714675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:46.714675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:46.714675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:46.714675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:46.714675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:46.714675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:46.714675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:46.714675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:46.714675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:46.714675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:46.714675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:46.714675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:46.714675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:46.714675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:46.714675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:46.716402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:46.716402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:46.716402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:46.716402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:46.716402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:46.716402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:46.716402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:46.716402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:46.716402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:46.716402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:46.716402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:46.716402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:46.716402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:46.716402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:46.716402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:46.716402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:46.717260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:46.717260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:46.717260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:46.717260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:46.717260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:46.717260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:46.717260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:46.717260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:46.717260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:46.717260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:46.717260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 20:21:46.717260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:46.717260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 476/971] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jul 14 20:21:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:46.717982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16fab1-6ab3-4cad-91af-474891ea8b60 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:46.750784 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:46.827033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16fab1-6ab3-4cad-91af-474891ea8b60 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:46.870156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92f14f-608f-4820-af20-df7fb9ff4d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:46.870156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92f14f-608f-4820-af20-df7fb9ff4d07 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:46.870439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92f14f-608f-4820-af20-df7fb9ff4d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:46.870726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92f14f-608f-4820-af20-df7fb9ff4d07 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:46.871478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92f14f-608f-4820-af20-df7fb9ff4d07 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:46.871924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92f14f-608f-4820-af20-df7fb9ff4d07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:46.873155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92f14f-608f-4820-af20-df7fb9ff4d07 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:46.881904 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 496/972] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 14 20:21:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:46.887632 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01dd04-60d9-48fb-ac2f-07aa33ad42f3 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:46.987969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:46.998903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01dd04-60d9-48fb-ac2f-07aa33ad42f3 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:47.033914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059ab8c-718a-488a-8005-4901be47c195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:47.034709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059ab8c-718a-488a-8005-4901be47c195 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:47.035123 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059ab8c-718a-488a-8005-4901be47c195 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:47.035320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059ab8c-718a-488a-8005-4901be47c195 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:47.046307 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059ab8c-718a-488a-8005-4901be47c195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:47.047797 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059ab8c-718a-488a-8005-4901be47c195 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:47.065354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-3059ab8c-718a-488a-8005-4901be47c195 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:47.065354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:47.065354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:47.065354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:47.065354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:47.065354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:47.065354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:47.065354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:47.065354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:47.065354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:47.065354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:47.065354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:47.065354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:47.065354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:47.065354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:47.066320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:47.066320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:47.066320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:47.066320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:47.066320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:47.066320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:47.066320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:47.066320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:47.066320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:47.066320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:47.066320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:47.066320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:47.066320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:47.066320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:47.066320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:47.066320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:47.066320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:47.067336 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:47.067336 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:47.067336 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:47.067336 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:47.067336 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 477/973] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:47.076467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92f14f-608f-4820-af20-df7fb9ff4d07 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:47.180308 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:47.188199 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92f14f-608f-4820-af20-df7fb9ff4d07 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:47.224104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0aa4b-d24e-4e81-9646-f57a28a6f3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:47.224865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0aa4b-d24e-4e81-9646-f57a28a6f3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:47.225220 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0aa4b-d24e-4e81-9646-f57a28a6f3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:47.225570 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0aa4b-d24e-4e81-9646-f57a28a6f3d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:47.226376 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0aa4b-d24e-4e81-9646-f57a28a6f3d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:47.226752 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0aa4b-d24e-4e81-9646-f57a28a6f3d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:47.228225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0aa4b-d24e-4e81-9646-f57a28a6f3d6 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:47.245063 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 497/974] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:47.246468 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059ab8c-718a-488a-8005-4901be47c195 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:47.356248 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059ab8c-718a-488a-8005-4901be47c195 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:47.391713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abecab2-ee0f-4af8-a3db-fb184c0f3d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:47.392631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abecab2-ee0f-4af8-a3db-fb184c0f3d36 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:47.392981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abecab2-ee0f-4af8-a3db-fb184c0f3d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:47.393465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abecab2-ee0f-4af8-a3db-fb184c0f3d36 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:47.401660 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abecab2-ee0f-4af8-a3db-fb184c0f3d36 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:47.403170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abecab2-ee0f-4af8-a3db-fb184c0f3d36 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:47.417534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-9abecab2-ee0f-4af8-a3db-fb184c0f3d36 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 20:21:47.417534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:47.417534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:47.417534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:47.417534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:47.417534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:47.417534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:47.417534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:47.417534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:47.417534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:47.417534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:47.417534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:47.417534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:47.417534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:47.417534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:47.417534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:47.418491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:47.418491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:47.418491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:47.418491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:47.418491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:47.418491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:47.418491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:47.418491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:47.418491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:47.418491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:47.418491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:47.418491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:47.418491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:47.418491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:47.418491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:47.418491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:47.418491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:47.418491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:47.420552 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:47.420552 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:47.420552 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:47.420552 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:47.420552 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:47.420552 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:47.420552 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:47.420552 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:47.420552 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 20:21:47.420552 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:47.423391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 478/975] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 14 20:21:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:47.424657 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0aa4b-d24e-4e81-9646-f57a28a6f3d6 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:47.528096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0aa4b-d24e-4e81-9646-f57a28a6f3d6 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:47.562770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cbb92-9170-496e-aee8-bc3e947000a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:47.563673 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cbb92-9170-496e-aee8-bc3e947000a7 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:47.563957 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cbb92-9170-496e-aee8-bc3e947000a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:47.564232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cbb92-9170-496e-aee8-bc3e947000a7 None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:47.565128 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65cbb92-9170-496e-aee8-bc3e947000a7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:47.565439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65cbb92-9170-496e-aee8-bc3e947000a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:47.566911 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65cbb92-9170-496e-aee8-bc3e947000a7 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:47.586814 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 498/976] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jul 14 20:21:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:47.593071 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abecab2-ee0f-4af8-a3db-fb184c0f3d36 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:47.596338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:47.707124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abecab2-ee0f-4af8-a3db-fb184c0f3d36 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:47.741196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4116e4-51fe-4ab0-aa19-b31ae7276e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:47.741844 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4116e4-51fe-4ab0-aa19-b31ae7276e86 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:47.742125 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4116e4-51fe-4ab0-aa19-b31ae7276e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:47.742395 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4116e4-51fe-4ab0-aa19-b31ae7276e86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:47.811949 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4116e4-51fe-4ab0-aa19-b31ae7276e86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:47.813306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4116e4-51fe-4ab0-aa19-b31ae7276e86 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:47.815213 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:47.919115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 479/977] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:21:47 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:21:47.921668 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cbb92-9170-496e-aee8-bc3e947000a7 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:47.938169 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:48.025557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cbb92-9170-496e-aee8-bc3e947000a7 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:48.061815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd6b61-a541-458e-b73f-0c768d43d76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:48.062670 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd6b61-a541-458e-b73f-0c768d43d76a None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:48.062892 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd6b61-a541-458e-b73f-0c768d43d76a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:48.063204 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd6b61-a541-458e-b73f-0c768d43d76a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:48.073728 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd6b61-a541-458e-b73f-0c768d43d76a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:48.075305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd6b61-a541-458e-b73f-0c768d43d76a None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:48.093503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-6bcd6b61-a541-458e-b73f-0c768d43d76a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:48.093503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:48.093503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:48.093503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:48.093503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:48.093503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:48.093503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:48.093503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:48.093503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:48.093503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:48.093503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:48.093503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:48.093503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:48.093503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:48.093503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:48.094499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:48.094499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:48.094499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:48.094499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:48.094499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:48.094499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:48.094499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:48.094499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:48.094499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:48.094499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:48.094499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:48.094499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:48.094499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:48.094499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:48.094499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:48.094499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:48.094499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:48.095833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:48.095833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:48.095833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:48.095833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:48.095833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 499/978] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:48.100532 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4116e4-51fe-4ab0-aa19-b31ae7276e86 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:48.204694 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4116e4-51fe-4ab0-aa19-b31ae7276e86 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:48.206262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:48.240255 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ef1f9-48db-45be-896a-8be320620a5b None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:48.240572 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ef1f9-48db-45be-896a-8be320620a5b None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:48.240834 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ef1f9-48db-45be-896a-8be320620a5b None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:48.242344 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 500/979] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:21:48.242449 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a383d-a806-4cf2-847c-579794a1ffe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:48.242955 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a383d-a806-4cf2-847c-579794a1ffe2 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:48.243320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a383d-a806-4cf2-847c-579794a1ffe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:48.243601 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a383d-a806-4cf2-847c-579794a1ffe2 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:48.244323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a383d-a806-4cf2-847c-579794a1ffe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:48.244627 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a383d-a806-4cf2-847c-579794a1ffe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:48.246066 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a383d-a806-4cf2-847c-579794a1ffe2 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:48.247492 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03604962-0a9b-41ab-9c54-a3a67b8891ae None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:48.247651 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03604962-0a9b-41ab-9c54-a3a67b8891ae None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:48.247907 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03604962-0a9b-41ab-9c54-a3a67b8891ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:48.260521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 480/980] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:48.265086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-03604962-0a9b-41ab-9c54-a3a67b8891ae None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:48.265980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a383d-a806-4cf2-847c-579794a1ffe2 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:48.266172 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-03604962-0a9b-41ab-9c54-a3a67b8891ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:48.330616 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:48.339644 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 501/981] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 20:21:48.362172 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03604962-0a9b-41ab-9c54-a3a67b8891ae None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:48.383785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a383d-a806-4cf2-847c-579794a1ffe2 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:48.423526 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c4c16-105b-43bf-9fea-a9aab71bb77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:48.424335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c4c16-105b-43bf-9fea-a9aab71bb77d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:48.424639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c4c16-105b-43bf-9fea-a9aab71bb77d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:48.424894 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c4c16-105b-43bf-9fea-a9aab71bb77d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:48.433622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c4c16-105b-43bf-9fea-a9aab71bb77d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:48.435137 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c4c16-105b-43bf-9fea-a9aab71bb77d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:48.454276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-d88c4c16-105b-43bf-9fea-a9aab71bb77d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:48.454276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:48.454276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:48.454276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:48.454276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:48.454276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:48.454276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:48.454276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:48.454276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:48.454276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:48.454276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:48.454276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:48.454276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:48.454276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:48.454276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:48.455318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:48.455318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:48.455318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:48.455318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:48.455318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:48.455318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:48.455318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:48.455318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:48.455318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:48.455318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:48.455318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:48.455318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:48.455318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:48.455318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:48.455318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:48.455318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:48.455318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:48.456376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:48.456376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:48.456376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:48.456376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:48.456376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:48.456376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:48.456376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:48.456376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:48.460226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 481/982] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:21:48.461527 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c4c16-105b-43bf-9fea-a9aab71bb77d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:48.474041 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03604962-0a9b-41ab-9c54-a3a67b8891ae None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:48.486418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:48.509169 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af07b3-be03-4e63-9ff6-e931403e9233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:48.509982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af07b3-be03-4e63-9ff6-e931403e9233 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:48.510262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af07b3-be03-4e63-9ff6-e931403e9233 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:48.510537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af07b3-be03-4e63-9ff6-e931403e9233 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:48.511221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af07b3-be03-4e63-9ff6-e931403e9233 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:48.513121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af07b3-be03-4e63-9ff6-e931403e9233 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:48.525333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-34af07b3-be03-4e63-9ff6-e931403e9233 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 20:21:48.525333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:48.525333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:48.525333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:48.525333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:48.525333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:48.525333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:48.525333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:48.525333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 20:21:48.525333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:48.525333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:48.525333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:48.525333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:48.525333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:48.525333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:48.525333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:48.525333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:48.526407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:48.526407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:48.526407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:48.526407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:48.526407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:48.526407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:48.526407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:48.526407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:48.526407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:48.526407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:48.526407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:48.526407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:48.526407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:48.526407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:48.526407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:48.526407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:48.527284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:48.527284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:48.527284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:48.527284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:48.527284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:48.527284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:48.527284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:48.527284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:48.527284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:48.527284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:48.527284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:48.527284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:48.527284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 20:21:48.527284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:48.527284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 502/983] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 14 20:21:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:21:48.528736 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43dcdd-7fa1-46a7-98f2-7134aaeb099b None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:48.528736 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43dcdd-7fa1-46a7-98f2-7134aaeb099b None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:48.529009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43dcdd-7fa1-46a7-98f2-7134aaeb099b None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:48.530405 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 503/984] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:48.532909 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43dcdd-7fa1-46a7-98f2-7134aaeb099b None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:48.571707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c4c16-105b-43bf-9fea-a9aab71bb77d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:48.593060 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:48.608131 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285b92d-e941-44f9-8948-f23af10a95ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:48.608862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285b92d-e941-44f9-8948-f23af10a95ff None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:48.609143 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285b92d-e941-44f9-8948-f23af10a95ff None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:48.609413 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285b92d-e941-44f9-8948-f23af10a95ff None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:48.610197 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285b92d-e941-44f9-8948-f23af10a95ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:48.610504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285b92d-e941-44f9-8948-f23af10a95ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:48.611959 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285b92d-e941-44f9-8948-f23af10a95ff None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:48.623804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 482/985] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:48.625024 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a3542-aec1-4923-b79c-e6f767a71d7e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:48.625305 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a3542-aec1-4923-b79c-e6f767a71d7e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:48.625576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a3542-aec1-4923-b79c-e6f767a71d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:48.642188 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43dcdd-7fa1-46a7-98f2-7134aaeb099b None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:48.644508 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1f4a3542-aec1-4923-b79c-e6f767a71d7e None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:48.646036 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1f4a3542-aec1-4923-b79c-e6f767a71d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:48.679561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d80d1-5ddc-4dcb-8949-67adec4032ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:48.680233 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d80d1-5ddc-4dcb-8949-67adec4032ca None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:48.680511 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d80d1-5ddc-4dcb-8949-67adec4032ca None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:48.680814 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d80d1-5ddc-4dcb-8949-67adec4032ca None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:48.681527 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d80d1-5ddc-4dcb-8949-67adec4032ca None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:48.681830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d80d1-5ddc-4dcb-8949-67adec4032ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:48.683225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d80d1-5ddc-4dcb-8949-67adec4032ca None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:48.688245 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 504/986] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 14 20:21:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:48.689126 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d80d1-5ddc-4dcb-8949-67adec4032ca None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:48.724623 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 483/987] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:48.725467 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a3542-aec1-4923-b79c-e6f767a71d7e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:48.740476 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:48.795895 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d80d1-5ddc-4dcb-8949-67adec4032ca None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:48.819735 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:48.832469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc4321-0b3a-498e-b3f1-3f4b4709cb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:48.833129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc4321-0b3a-498e-b3f1-3f4b4709cb2a None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:48.833380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a3542-aec1-4923-b79c-e6f767a71d7e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:48.833491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc4321-0b3a-498e-b3f1-3f4b4709cb2a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:48.833673 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc4321-0b3a-498e-b3f1-3f4b4709cb2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:48.834503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc4321-0b3a-498e-b3f1-3f4b4709cb2a None admin] RBAC: Adding query filter params (role.id=f932ef767ec247d8ad3823e16f1b0306, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=4f75b8b484384554ad59aa20535875e9) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:48.834815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc4321-0b3a-498e-b3f1-3f4b4709cb2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:48.836276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc4321-0b3a-498e-b3f1-3f4b4709cb2a None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:48.842311 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 505/988] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:21:48 2022] GET /identity/v3/role_assignments?user.id=4f75b8b484384554ad59aa20535875e9&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=f932ef767ec247d8ad3823e16f1b0306 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:48.843214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc4321-0b3a-498e-b3f1-3f4b4709cb2a None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:48.884192 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfafa7d-845e-4465-bb90-a067b7248d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:48.884871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfafa7d-845e-4465-bb90-a067b7248d97 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:48.885155 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfafa7d-845e-4465-bb90-a067b7248d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:48.885430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfafa7d-845e-4465-bb90-a067b7248d97 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:48.886157 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfafa7d-845e-4465-bb90-a067b7248d97 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:48.886461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfafa7d-845e-4465-bb90-a067b7248d97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:48.888499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfafa7d-845e-4465-bb90-a067b7248d97 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:48.899492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 484/989] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 14 20:21:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:48.900746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfafa7d-845e-4465-bb90-a067b7248d97 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:48.963349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc4321-0b3a-498e-b3f1-3f4b4709cb2a None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:48.989493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:49.004285 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78e946-2ef7-44ce-a097-304155da6d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:49.005817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78e946-2ef7-44ce-a097-304155da6d1d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:49.005817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78e946-2ef7-44ce-a097-304155da6d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:49.005817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78e946-2ef7-44ce-a097-304155da6d1d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:49.007805 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78e946-2ef7-44ce-a097-304155da6d1d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:49.007805 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78e946-2ef7-44ce-a097-304155da6d1d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:49.021642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-ba78e946-2ef7-44ce-a097-304155da6d1d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 20:21:49.021642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:49.021642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:49.021642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:49.021642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:49.021642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:49.021642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:49.021642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:49.021642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 20:21:49.021642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:49.021642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:49.021642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:49.021642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:49.021642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:49.021642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:49.021642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:49.021642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:49.022698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:49.022698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:49.022698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:49.022698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:49.022698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:49.022698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:49.022698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:49.022698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:49.022698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:49.022698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:49.022698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:49.022698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:49.022698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:49.022698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:49.022698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:49.022698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:49.023713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:49.023713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:49.023713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:49.023713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:49.023713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:49.023713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:49.023713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:49.023713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:49.023713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:49.023713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:49.023713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:49.023713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:49.023713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 20:21:49.023713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:49.025897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 506/990] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:49.026333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfafa7d-845e-4465-bb90-a067b7248d97 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:49.033104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78e946-2ef7-44ce-a097-304155da6d1d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:49.068871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb494f-1528-4764-a14a-e2628956b866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:49.069539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb494f-1528-4764-a14a-e2628956b866 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:49.069827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb494f-1528-4764-a14a-e2628956b866 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:49.070106 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb494f-1528-4764-a14a-e2628956b866 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:49.070906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb494f-1528-4764-a14a-e2628956b866 None admin] RBAC: Adding query filter params (interface=admin, service_id=9f7c0bba4a7743b593449bb3e84430bd, region_id=RegionOne) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:49.071988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb494f-1528-4764-a14a-e2628956b866 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:49.072660 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb494f-1528-4764-a14a-e2628956b866 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:49.079522 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 485/991] 10.222.0.52 () {56 vars in 1249 bytes} [Thu Jul 14 20:21:48 2022] GET /identity/v3/endpoints?service_id=9f7c0bba4a7743b593449bb3e84430bd&interface=admin®ion_id=RegionOne => generated 193 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:49.084708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb494f-1528-4764-a14a-e2628956b866 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:49.091823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:49.144915 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78e946-2ef7-44ce-a097-304155da6d1d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:49.181897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27752eb-7ee7-4740-84a5-8d486f41025d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:49.182567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27752eb-7ee7-4740-84a5-8d486f41025d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:49.182852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27752eb-7ee7-4740-84a5-8d486f41025d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:49.183159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27752eb-7ee7-4740-84a5-8d486f41025d None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:49.183938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27752eb-7ee7-4740-84a5-8d486f41025d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:49.184750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27752eb-7ee7-4740-84a5-8d486f41025d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:49.186128 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27752eb-7ee7-4740-84a5-8d486f41025d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:49.191166 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb494f-1528-4764-a14a-e2628956b866 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:49.191587 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 507/992] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 14 20:21:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:49.197312 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27752eb-7ee7-4740-84a5-8d486f41025d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:49.233679 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce59361-fa87-4df4-a051-e276a161e403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:49.234354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce59361-fa87-4df4-a051-e276a161e403 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:49.234642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce59361-fa87-4df4-a051-e276a161e403 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:49.234922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce59361-fa87-4df4-a051-e276a161e403 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:49.235859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce59361-fa87-4df4-a051-e276a161e403 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:49.237054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce59361-fa87-4df4-a051-e276a161e403 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:49.244276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 486/993] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 14 20:21:49 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:21:49.312473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27752eb-7ee7-4740-84a5-8d486f41025d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:49.331300 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:49.352764 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150ed93-c0c1-4bdb-bf56-cadec141479f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:49.353484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150ed93-c0c1-4bdb-bf56-cadec141479f None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:49.353726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150ed93-c0c1-4bdb-bf56-cadec141479f None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:49.354001 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150ed93-c0c1-4bdb-bf56-cadec141479f None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:49.354722 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150ed93-c0c1-4bdb-bf56-cadec141479f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:49.355064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150ed93-c0c1-4bdb-bf56-cadec141479f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:49.357033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150ed93-c0c1-4bdb-bf56-cadec141479f None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:49.367309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 508/994] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 14 20:21:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:49.369391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce59361-fa87-4df4-a051-e276a161e403 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:49.375593 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:49.480001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce59361-fa87-4df4-a051-e276a161e403 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:49.516675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a9612-6651-4cb1-86f1-5c233c41ffc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:49.517713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a9612-6651-4cb1-86f1-5c233c41ffc0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:49.518199 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a9612-6651-4cb1-86f1-5c233c41ffc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:49.518694 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a9612-6651-4cb1-86f1-5c233c41ffc0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:49.519977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a9612-6651-4cb1-86f1-5c233c41ffc0 None admin] RBAC: Adding query filter params (interface=public, service_id=3c78cffb2ba44eb0a6bf5b600741497f, region_id=RegionOne) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:49.520511 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a9612-6651-4cb1-86f1-5c233c41ffc0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:49.522197 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a9612-6651-4cb1-86f1-5c233c41ffc0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:49.528100 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 487/995] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 14 20:21:49 2022] GET /identity/v3/endpoints?service_id=3c78cffb2ba44eb0a6bf5b600741497f&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:49.533831 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150ed93-c0c1-4bdb-bf56-cadec141479f None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:49.647390 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150ed93-c0c1-4bdb-bf56-cadec141479f None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:49.684390 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdeaa25-43a4-44c7-bfe4-dd51890996bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:49.685089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeaa25-43a4-44c7-bfe4-dd51890996bf None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:49.685370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeaa25-43a4-44c7-bfe4-dd51890996bf None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:49.685636 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeaa25-43a4-44c7-bfe4-dd51890996bf None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:49.686317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdeaa25-43a4-44c7-bfe4-dd51890996bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:49.687754 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdeaa25-43a4-44c7-bfe4-dd51890996bf None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:49.694779 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 509/996] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 14 20:21:49 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:21:50.897798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82557e43-2f07-40a7-a69d-2bb7f5a6d02f None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:50.898297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82557e43-2f07-40a7-a69d-2bb7f5a6d02f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:50.898297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82557e43-2f07-40a7-a69d-2bb7f5a6d02f None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:50.899807 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 488/997] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:50.905089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ab79a-0504-43d0-b936-d77e601648f4 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:50.905505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ab79a-0504-43d0-b936-d77e601648f4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:50.905579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ab79a-0504-43d0-b936-d77e601648f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:50.923036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-662ab79a-0504-43d0-b936-d77e601648f4 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:50.924231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-662ab79a-0504-43d0-b936-d77e601648f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:50.966095 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:50.995106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 510/998] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:51.017290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82557e43-2f07-40a7-a69d-2bb7f5a6d02f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:51.086295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:51.124634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82557e43-2f07-40a7-a69d-2bb7f5a6d02f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:51.160717 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04373a-8a7b-403a-b970-11ab09116f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:51.161425 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04373a-8a7b-403a-b970-11ab09116f45 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:51.161709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04373a-8a7b-403a-b970-11ab09116f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:51.161983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04373a-8a7b-403a-b970-11ab09116f45 None admin] PATH_INFO: `/v3/services/image` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:51.162635 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04373a-8a7b-403a-b970-11ab09116f45 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:51.164039 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04373a-8a7b-403a-b970-11ab09116f45 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:51.176226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-5a04373a-8a7b-403a-b970-11ab09116f45 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 20:21:51.176226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:51.176226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:51.176226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:51.176226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:51.176226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:51.176226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:51.176226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:51.176226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 20:21:51.176226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:51.176226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:51.176226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:51.176226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:51.176226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:51.176226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:51.176226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:51.176226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:51.177278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:51.177278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:51.177278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:51.177278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:51.177278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:51.177278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:51.177278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:51.177278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:51.177278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:51.177278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:51.177278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:51.177278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:51.177278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:51.177278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:51.177278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:51.177278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:51.178119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:51.178119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:51.178119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:51.178119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:51.178119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:51.178119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:51.178119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:51.178119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:51.178119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:51.178119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:51.178119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:51.178119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:51.178119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 20:21:51.178119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:51.182287 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 489/999] 10.222.0.52 () {56 vars in 1105 bytes} [Thu Jul 14 20:21:51 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:51.183574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ab79a-0504-43d0-b936-d77e601648f4 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:51.269167 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa0eda-9cdb-4b3e-8d69-837397fe878f None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:51.269493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa0eda-9cdb-4b3e-8d69-837397fe878f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:51.269734 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa0eda-9cdb-4b3e-8d69-837397fe878f None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:51.271272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 490/1000] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:51.276379 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd7a02-37ab-4428-b8aa-7f83831006c6 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:51.276684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd7a02-37ab-4428-b8aa-7f83831006c6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:51.276938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd7a02-37ab-4428-b8aa-7f83831006c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:51.289327 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ab79a-0504-43d0-b936-d77e601648f4 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:51.295550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1efd7a02-37ab-4428-b8aa-7f83831006c6 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:51.296751 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1efd7a02-37ab-4428-b8aa-7f83831006c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:51.328614 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771df5d7-fde3-4ef4-8866-40295b814446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:51.329287 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771df5d7-fde3-4ef4-8866-40295b814446 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:51.329578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771df5d7-fde3-4ef4-8866-40295b814446 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:51.329857 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771df5d7-fde3-4ef4-8866-40295b814446 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:51.330568 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771df5d7-fde3-4ef4-8866-40295b814446 None admin] RBAC: Adding query filter params (name=image) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:51.330879 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771df5d7-fde3-4ef4-8866-40295b814446 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:51.332285 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771df5d7-fde3-4ef4-8866-40295b814446 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:51.343716 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 511/1001] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:51.345718 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771df5d7-fde3-4ef4-8866-40295b814446 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:51.375131 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 491/1002] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 20:21:51.393363 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd7a02-37ab-4428-b8aa-7f83831006c6 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:51.396803 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:51.459769 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:51.461554 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771df5d7-fde3-4ef4-8866-40295b814446 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:51.497438 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ba463-56a5-4271-8ac8-114c64ef94f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:51.498111 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ba463-56a5-4271-8ac8-114c64ef94f3 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:51.498396 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ba463-56a5-4271-8ac8-114c64ef94f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:51.498677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ba463-56a5-4271-8ac8-114c64ef94f3 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:51.499417 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ba463-56a5-4271-8ac8-114c64ef94f3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:51.499721 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ba463-56a5-4271-8ac8-114c64ef94f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:51.501082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ba463-56a5-4271-8ac8-114c64ef94f3 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:51.505790 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 512/1003] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:51 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:51.506128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd7a02-37ab-4428-b8aa-7f83831006c6 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:51.554455 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a0b7c-f2f2-4b1d-a89a-83385e455521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:51.555324 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a0b7c-f2f2-4b1d-a89a-83385e455521 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:51.555613 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a0b7c-f2f2-4b1d-a89a-83385e455521 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:51.555891 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a0b7c-f2f2-4b1d-a89a-83385e455521 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:51.556592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a0b7c-f2f2-4b1d-a89a-83385e455521 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:51.557991 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a0b7c-f2f2-4b1d-a89a-83385e455521 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:51.571465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-091a0b7c-f2f2-4b1d-a89a-83385e455521 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 20:21:51.571465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:51.571465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:51.571465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:51.571465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:51.571465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:51.571465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:51.571465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:51.571465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 20:21:51.571465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:51.571465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:51.571465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:51.571465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:51.571465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:51.571465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:51.571465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:51.571465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:51.572520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:51.572520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:51.572520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:51.572520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:51.572520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:51.572520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:51.572520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:51.572520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:51.572520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:51.572520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:51.572520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:51.572520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:51.572520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:51.572520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:51.572520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:51.572520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:51.573391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:51.573391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:51.573391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:51.573391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:51.573391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:51.573391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:51.573391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:51.573391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:51.573391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:51.573391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:51.573391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:51.573391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:51.573391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 20:21:51.573391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:51.578598 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 492/1004] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 14 20:21:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:21:51.580149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ba463-56a5-4271-8ac8-114c64ef94f3 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:51.709335 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ba463-56a5-4271-8ac8-114c64ef94f3 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:51.711156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:51.728352 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b4ad0-d1ad-4029-9d93-4a90924b0e8e None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:51.728655 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b4ad0-d1ad-4029-9d93-4a90924b0e8e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:51.728929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b4ad0-d1ad-4029-9d93-4a90924b0e8e None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:51.730407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 493/1005] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:51.735217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa166dd5-1a9b-43dc-92f3-0aa0b2c6e1f7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:51.735505 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa166dd5-1a9b-43dc-92f3-0aa0b2c6e1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:51.735833 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa166dd5-1a9b-43dc-92f3-0aa0b2c6e1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:51.747285 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2da4e-7fac-4fd9-ae32-b8086320d797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:51.747952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2da4e-7fac-4fd9-ae32-b8086320d797 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:51.748232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2da4e-7fac-4fd9-ae32-b8086320d797 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:51.748510 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2da4e-7fac-4fd9-ae32-b8086320d797 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:51.749216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2da4e-7fac-4fd9-ae32-b8086320d797 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:51.749521 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2da4e-7fac-4fd9-ae32-b8086320d797 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:51.750881 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2da4e-7fac-4fd9-ae32-b8086320d797 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:51.752997 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-fa166dd5-1a9b-43dc-92f3-0aa0b2c6e1f7 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:51.754248 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fa166dd5-1a9b-43dc-92f3-0aa0b2c6e1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:51.756598 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 513/1006] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 14 20:21:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:51.761766 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2da4e-7fac-4fd9-ae32-b8086320d797 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:51.833344 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 494/1007] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 20:21:51.856321 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa166dd5-1a9b-43dc-92f3-0aa0b2c6e1f7 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:51.874079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2da4e-7fac-4fd9-ae32-b8086320d797 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:51.912051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:51.914888 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a8895-9335-44ba-8557-a381ca474790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:51.916817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a8895-9335-44ba-8557-a381ca474790 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:51.917244 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a8895-9335-44ba-8557-a381ca474790 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:51.917763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a8895-9335-44ba-8557-a381ca474790 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:51.918782 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a8895-9335-44ba-8557-a381ca474790 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:51.919283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a8895-9335-44ba-8557-a381ca474790 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:51.920783 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a8895-9335-44ba-8557-a381ca474790 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:51.926498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 514/1008] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 14 20:21:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:51.932158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a8895-9335-44ba-8557-a381ca474790 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:51.963158 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa166dd5-1a9b-43dc-92f3-0aa0b2c6e1f7 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:51.965131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:51.999959 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2815c1-2f9b-45e2-8091-3563c46186ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:52.000863 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2815c1-2f9b-45e2-8091-3563c46186ad None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:52.001142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2815c1-2f9b-45e2-8091-3563c46186ad None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:52.001414 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2815c1-2f9b-45e2-8091-3563c46186ad None admin] PATH_INFO: `/v3/services/compute` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:52.002067 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2815c1-2f9b-45e2-8091-3563c46186ad None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:52.003457 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2815c1-2f9b-45e2-8091-3563c46186ad None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:52.015800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-9b2815c1-2f9b-45e2-8091-3563c46186ad None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 20:21:52.015800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:52.015800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:52.015800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:52.015800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:52.015800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:52.015800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:52.015800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:52.015800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 20:21:52.015800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:52.015800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:52.015800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:52.015800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:52.015800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:52.015800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:52.015800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:52.015800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:52.016936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:52.016936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:52.016936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:52.016936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:52.016936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:52.016936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:52.016936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:52.016936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:52.016936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:52.016936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:52.016936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:52.016936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:52.016936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:52.016936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:52.016936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:52.016936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:52.017766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:52.017766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:52.017766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:52.017766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:52.017766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:52.017766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:52.017766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:52.017766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:52.017766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:52.017766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:52.017766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:52.017766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:52.017766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 20:21:52.017766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:52.017766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 495/1009] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:52.024752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2815c1-2f9b-45e2-8091-3563c46186ad None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:52.055078 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a8895-9335-44ba-8557-a381ca474790 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:52.093950 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d47e09-0c6c-4274-8d86-e2de547fdbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:52.094604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d47e09-0c6c-4274-8d86-e2de547fdbe5 None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:52.094891 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d47e09-0c6c-4274-8d86-e2de547fdbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:52.095191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d47e09-0c6c-4274-8d86-e2de547fdbe5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:52.095925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d47e09-0c6c-4274-8d86-e2de547fdbe5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:52.097590 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d47e09-0c6c-4274-8d86-e2de547fdbe5 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:52.113144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 515/1010] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 14 20:21:51 2022] POST /identity/v3/endpoints => generated 325 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:21:52.135619 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2815c1-2f9b-45e2-8091-3563c46186ad None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:52.159066 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:52.184407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c80ae1-7343-4442-af99-b598f4b3db7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:52.185099 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c80ae1-7343-4442-af99-b598f4b3db7f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:52.185383 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c80ae1-7343-4442-af99-b598f4b3db7f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:52.185661 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c80ae1-7343-4442-af99-b598f4b3db7f None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:52.186391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c80ae1-7343-4442-af99-b598f4b3db7f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:52.186693 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c80ae1-7343-4442-af99-b598f4b3db7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:52.188160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c80ae1-7343-4442-af99-b598f4b3db7f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:52.193990 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 496/1011] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 14 20:21:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:52.200117 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d47e09-0c6c-4274-8d86-e2de547fdbe5 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:52.303424 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:52.316435 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d47e09-0c6c-4274-8d86-e2de547fdbe5 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:52.354747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77678aa-e4ab-4c2e-8469-e8ba247221c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:52.356407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77678aa-e4ab-4c2e-8469-e8ba247221c9 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:52.356943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77678aa-e4ab-4c2e-8469-e8ba247221c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:52.357416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77678aa-e4ab-4c2e-8469-e8ba247221c9 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:52.358348 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77678aa-e4ab-4c2e-8469-e8ba247221c9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:52.358862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77678aa-e4ab-4c2e-8469-e8ba247221c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:52.360485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77678aa-e4ab-4c2e-8469-e8ba247221c9 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:52.366331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 516/1012] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 14 20:21:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:52.373138 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c80ae1-7343-4442-af99-b598f4b3db7f None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:52.483311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c80ae1-7343-4442-af99-b598f4b3db7f None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:52.518444 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965daae4-4ea8-4931-a785-e2ba837bd4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:52.519385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965daae4-4ea8-4931-a785-e2ba837bd4c6 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:52.519669 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965daae4-4ea8-4931-a785-e2ba837bd4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:52.519940 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965daae4-4ea8-4931-a785-e2ba837bd4c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:52.520680 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965daae4-4ea8-4931-a785-e2ba837bd4c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:52.522035 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965daae4-4ea8-4931-a785-e2ba837bd4c6 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:52.540225 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 497/1013] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 14 20:21:52 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:21:53.568318 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a67db1-f85f-4366-a414-e815bd76f931 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:53.568829 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a67db1-f85f-4366-a414-e815bd76f931 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:53.568935 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a67db1-f85f-4366-a414-e815bd76f931 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:53.570222 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 517/1014] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:53.576222 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ac4e9-80fb-41e0-b059-1684ba17bea5 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:53.576707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ac4e9-80fb-41e0-b059-1684ba17bea5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:53.576786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ac4e9-80fb-41e0-b059-1684ba17bea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:53.594541 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-248ac4e9-80fb-41e0-b059-1684ba17bea5 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:53.595782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-248ac4e9-80fb-41e0-b059-1684ba17bea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:53.645785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:53.697397 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 498/1015] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:53 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:53.721064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a67db1-f85f-4366-a414-e815bd76f931 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:53.830073 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a67db1-f85f-4366-a414-e815bd76f931 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:53.864669 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd5e5d-6bca-4570-a38b-bc42df06ed81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:53.865334 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd5e5d-6bca-4570-a38b-bc42df06ed81 None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:53.865653 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd5e5d-6bca-4570-a38b-bc42df06ed81 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:53.865889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd5e5d-6bca-4570-a38b-bc42df06ed81 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:53.866621 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd5e5d-6bca-4570-a38b-bc42df06ed81 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:53.868020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd5e5d-6bca-4570-a38b-bc42df06ed81 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:53.890982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 518/1016] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 14 20:21:53 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:21:54.196477 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8dba2-8619-400a-8932-c52046d4ef77 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:54.196668 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8dba2-8619-400a-8932-c52046d4ef77 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:54.196977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8dba2-8619-400a-8932-c52046d4ef77 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:54.198486 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 499/1017] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:54.203996 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d8234-5166-4123-bb13-4be3cd9c8f38 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:54.204284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d8234-5166-4123-bb13-4be3cd9c8f38 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:54.204550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d8234-5166-4123-bb13-4be3cd9c8f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:54.222240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-e63d8234-5166-4123-bb13-4be3cd9c8f38 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:54.223470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e63d8234-5166-4123-bb13-4be3cd9c8f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:54.234169 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:54.312473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 519/1018] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:54.335637 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8dba2-8619-400a-8932-c52046d4ef77 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:54.454406 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8dba2-8619-400a-8932-c52046d4ef77 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:54.489557 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d40f8-1c2e-4dbc-8e3d-2f7f0e8a9847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:54.490341 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d40f8-1c2e-4dbc-8e3d-2f7f0e8a9847 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:54.490618 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d40f8-1c2e-4dbc-8e3d-2f7f0e8a9847 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:54.490887 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d40f8-1c2e-4dbc-8e3d-2f7f0e8a9847 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:54.492074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d40f8-1c2e-4dbc-8e3d-2f7f0e8a9847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:54.493573 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d40f8-1c2e-4dbc-8e3d-2f7f0e8a9847 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:54.495430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 500/1019] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:54.501010 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d8234-5166-4123-bb13-4be3cd9c8f38 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:54.604151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d8234-5166-4123-bb13-4be3cd9c8f38 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:54.643038 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719db1b-d7b2-44e2-bc30-a18637209ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:54.643687 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719db1b-d7b2-44e2-bc30-a18637209ade None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:54.644238 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719db1b-d7b2-44e2-bc30-a18637209ade None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:54.644522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719db1b-d7b2-44e2-bc30-a18637209ade None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:54.645258 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719db1b-d7b2-44e2-bc30-a18637209ade None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:54.646695 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719db1b-d7b2-44e2-bc30-a18637209ade None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:54.659243 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:54.668937 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65099e40-308d-441d-8e29-9138c1a8caa9 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:54.669469 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65099e40-308d-441d-8e29-9138c1a8caa9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:54.669469 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65099e40-308d-441d-8e29-9138c1a8caa9 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:54.671174 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 501/1020] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:54.682974 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 520/1021] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 14 20:21:54 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:21:54.683184 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2eefc-bc31-4652-83f8-f6ac44d004bd None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:54.683184 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2eefc-bc31-4652-83f8-f6ac44d004bd None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:54.683184 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2eefc-bc31-4652-83f8-f6ac44d004bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:54.700854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a4b2eefc-bc31-4652-83f8-f6ac44d004bd None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:54.702230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a4b2eefc-bc31-4652-83f8-f6ac44d004bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:54.735767 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:54.791041 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 502/1022] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:54.811474 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719db1b-d7b2-44e2-bc30-a18637209ade None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:54.926217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719db1b-d7b2-44e2-bc30-a18637209ade None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:54.962820 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a71574-c385-4f85-a596-e2f0d00ffba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:54.963687 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a71574-c385-4f85-a596-e2f0d00ffba8 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:54.963973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a71574-c385-4f85-a596-e2f0d00ffba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:54.964251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a71574-c385-4f85-a596-e2f0d00ffba8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:54.975240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a71574-c385-4f85-a596-e2f0d00ffba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:54.983574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-e2a71574-c385-4f85-a596-e2f0d00ffba8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 20:21:54.983574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:54.983574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:54.983574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:54.983574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:54.983574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:54.983574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:54.983574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:54.983574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:54.983574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:54.983574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:54.983574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:54.983574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:54.983574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:54.983574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:54.983574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:54.984569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:54.984569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:54.984569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:54.984569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:54.984569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:54.984569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:54.984569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:54.984569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:54.984569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:54.984569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:54.984569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:54.984569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:54.984569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:54.984569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:54.984569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:54.984569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:54.985412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:54.985412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:54.985412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:54.985412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:54.985412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:54.985412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:54.985412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:54.985412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:54.985412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:54.985412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:54.985412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 20:21:54.985412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:54.985412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 521/1023] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jul 14 20:21:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:54.993007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2eefc-bc31-4652-83f8-f6ac44d004bd None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:55.109069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2eefc-bc31-4652-83f8-f6ac44d004bd None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:55.144791 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce780a-a3d0-4331-a012-4edf33effb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:55.145462 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce780a-a3d0-4331-a012-4edf33effb7e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:55.145748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce780a-a3d0-4331-a012-4edf33effb7e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:55.146026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce780a-a3d0-4331-a012-4edf33effb7e None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:55.146779 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce780a-a3d0-4331-a012-4edf33effb7e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:55.147118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce780a-a3d0-4331-a012-4edf33effb7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:55.148498 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce780a-a3d0-4331-a012-4edf33effb7e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:55.161772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 503/1024] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 14 20:21:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:55.163152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a71574-c385-4f85-a596-e2f0d00ffba8 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:55.221661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:55.273037 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a71574-c385-4f85-a596-e2f0d00ffba8 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:55.307720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902f0d3-a67b-4009-bf16-cfe036e97506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:55.308532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902f0d3-a67b-4009-bf16-cfe036e97506 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:55.308941 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902f0d3-a67b-4009-bf16-cfe036e97506 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:55.309152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902f0d3-a67b-4009-bf16-cfe036e97506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:55.320056 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902f0d3-a67b-4009-bf16-cfe036e97506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:55.321506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902f0d3-a67b-4009-bf16-cfe036e97506 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:55.340977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-9902f0d3-a67b-4009-bf16-cfe036e97506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:55.340977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:55.340977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:55.340977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:55.340977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:55.340977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:55.340977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:55.340977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:55.340977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:55.340977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:55.340977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:55.340977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:55.340977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:55.340977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:55.340977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:55.342057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:55.342057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:55.342057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:55.342057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:55.342057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:55.342057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:55.342057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:55.342057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:55.342057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:55.342057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:55.342057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:55.342057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:55.342057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:55.342057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:55.342057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:55.342057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:55.342057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:55.347894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:55.347894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:55.347894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:55.347894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:55.347894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 522/1025] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:55.348188 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce780a-a3d0-4331-a012-4edf33effb7e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:55.376435 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:55.460133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce780a-a3d0-4331-a012-4edf33effb7e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:55.495266 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21210c74-5684-47ec-ae09-f18075702b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:55.495941 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21210c74-5684-47ec-ae09-f18075702b30 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:55.496264 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21210c74-5684-47ec-ae09-f18075702b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:55.496543 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21210c74-5684-47ec-ae09-f18075702b30 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:55.497277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21210c74-5684-47ec-ae09-f18075702b30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:55.497588 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21210c74-5684-47ec-ae09-f18075702b30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:55.498988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21210c74-5684-47ec-ae09-f18075702b30 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:55.512727 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 504/1026] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:55.518490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902f0d3-a67b-4009-bf16-cfe036e97506 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:55.623999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902f0d3-a67b-4009-bf16-cfe036e97506 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:55.626726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:55.667174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045a327-e4b8-4679-9283-1a9019e2ed80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:55.667970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045a327-e4b8-4679-9283-1a9019e2ed80 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:55.668288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045a327-e4b8-4679-9283-1a9019e2ed80 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:55.668560 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045a327-e4b8-4679-9283-1a9019e2ed80 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:55.676585 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045a327-e4b8-4679-9283-1a9019e2ed80 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:55.678033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045a327-e4b8-4679-9283-1a9019e2ed80 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:55.693864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-7045a327-e4b8-4679-9283-1a9019e2ed80 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 20:21:55.693864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:55.693864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:55.693864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:55.693864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:55.693864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:55.693864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:55.693864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:55.693864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:55.693864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:55.693864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:55.693864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:55.693864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:21:55.693864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:21:55.693864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:21:55.693864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:21:55.700498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:55.700498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:55.700498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:21:55.700498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:55.700498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:21:55.700498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:21:55.700498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:55.700498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:55.700498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:55.700498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:55.700498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:55.700498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:55.700498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:55.700498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:55.700498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:55.700498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:55.700498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:55.700498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:55.702387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:55.702387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:55.702387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:21:55.702387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:21:55.702387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:21:55.702387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:21:55.702387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:21:55.702387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:21:55.702387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 20:21:55.702387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:55.702387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 523/1027] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 14 20:21:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:21:55.707313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21210c74-5684-47ec-ae09-f18075702b30 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:55.814143 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:55.840473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21210c74-5684-47ec-ae09-f18075702b30 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:55.881688 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03df7c-8681-4d98-81c4-6d209b38e54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:55.882295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03df7c-8681-4d98-81c4-6d209b38e54b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:55.882579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03df7c-8681-4d98-81c4-6d209b38e54b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:55.882856 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03df7c-8681-4d98-81c4-6d209b38e54b None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:55.883739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03df7c-8681-4d98-81c4-6d209b38e54b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:55.884048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03df7c-8681-4d98-81c4-6d209b38e54b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:55.885497 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03df7c-8681-4d98-81c4-6d209b38e54b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:55.910121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 505/1028] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 14 20:21:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:55.911738 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045a327-e4b8-4679-9283-1a9019e2ed80 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:55.945923 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921ae4b-eb56-4005-bb51-f7b64f63aa76 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:55.946254 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921ae4b-eb56-4005-bb51-f7b64f63aa76 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:55.946604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921ae4b-eb56-4005-bb51-f7b64f63aa76 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:55.948224 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 506/1029] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:21:55.953948 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948389e6-67ce-4a84-9caf-1a0add33769b None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:55.954510 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948389e6-67ce-4a84-9caf-1a0add33769b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:55.955040 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948389e6-67ce-4a84-9caf-1a0add33769b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:55.979616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-948389e6-67ce-4a84-9caf-1a0add33769b None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:55.981386 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-948389e6-67ce-4a84-9caf-1a0add33769b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:56.028214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045a327-e4b8-4679-9283-1a9019e2ed80 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:56.057625 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 507/1030] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 20:21:56.072018 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6cfbd-0c29-4d1e-a34b-0ec9fd4f0161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:56.072686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6cfbd-0c29-4d1e-a34b-0ec9fd4f0161 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:56.072969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6cfbd-0c29-4d1e-a34b-0ec9fd4f0161 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:56.073249 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6cfbd-0c29-4d1e-a34b-0ec9fd4f0161 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:56.080450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948389e6-67ce-4a84-9caf-1a0add33769b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:56.112510 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:56.154471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6cfbd-0c29-4d1e-a34b-0ec9fd4f0161 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:56.156003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6cfbd-0c29-4d1e-a34b-0ec9fd4f0161 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:56.165851 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:56.206090 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948389e6-67ce-4a84-9caf-1a0add33769b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:56.247823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381999c-803f-4afe-a4fb-6ac3c032e329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:56.248627 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381999c-803f-4afe-a4fb-6ac3c032e329 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:56.248900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381999c-803f-4afe-a4fb-6ac3c032e329 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:56.249177 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381999c-803f-4afe-a4fb-6ac3c032e329 None admin] PATH_INFO: `/v3/services/image` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:56.249825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381999c-803f-4afe-a4fb-6ac3c032e329 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:56.251236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381999c-803f-4afe-a4fb-6ac3c032e329 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:56.265524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-8381999c-803f-4afe-a4fb-6ac3c032e329 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 20:21:56.265524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:56.265524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:21:56.265524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:21:56.265524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:21:56.265524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:21:56.265524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:21:56.265524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:56.265524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 20:21:56.265524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:56.265524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:21:56.265524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:56.265524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:56.265524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:56.265524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:56.265524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:56.265524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:56.266605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:56.266605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:56.266605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:56.266605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:56.266605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:56.266605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:56.266605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:21:56.266605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:21:56.266605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:21:56.266605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:21:56.266605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:56.266605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:56.266605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:56.266605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:56.266605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:56.266605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:56.267952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:56.267952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:56.267952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:56.267952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:56.267952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:56.267952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:56.267952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:56.267952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:56.267952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:56.267952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:56.267952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:21:56.267952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:21:56.267952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 20:21:56.267952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:21:56.267952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 508/1031] 10.222.0.52 () {56 vars in 1105 bytes} [Thu Jul 14 20:21:56 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:21:56.273331 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381999c-803f-4afe-a4fb-6ac3c032e329 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:56.280151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 524/1032] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:21:55 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 20:21:56.281108 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6cfbd-0c29-4d1e-a34b-0ec9fd4f0161 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:56.352940 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:56.395937 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6cfbd-0c29-4d1e-a34b-0ec9fd4f0161 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:56.400832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381999c-803f-4afe-a4fb-6ac3c032e329 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:56.437555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03f44f-39ed-4070-b8ea-ce169502236a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:56.438250 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03f44f-39ed-4070-b8ea-ce169502236a None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:56.438535 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03f44f-39ed-4070-b8ea-ce169502236a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:56.438811 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03f44f-39ed-4070-b8ea-ce169502236a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:56.440623 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e30b5-4ea4-491e-93d5-61cde18a31a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:56.441294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e30b5-4ea4-491e-93d5-61cde18a31a1 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:56.441578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e30b5-4ea4-491e-93d5-61cde18a31a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:56.441856 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e30b5-4ea4-491e-93d5-61cde18a31a1 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:56.442559 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e30b5-4ea4-491e-93d5-61cde18a31a1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:56.442873 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e30b5-4ea4-491e-93d5-61cde18a31a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:56.444245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e30b5-4ea4-491e-93d5-61cde18a31a1 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:56.445731 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:56.450825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03f44f-39ed-4070-b8ea-ce169502236a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:56.455013 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 509/1033] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:56.455190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03f44f-39ed-4070-b8ea-ce169502236a None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:56.458840 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e30b5-4ea4-491e-93d5-61cde18a31a1 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:56.474989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-3b03f44f-39ed-4070-b8ea-ce169502236a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:56.474989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:56.474989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:56.474989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:56.474989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:56.474989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:56.474989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:56.474989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:56.474989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:56.474989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:56.474989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:56.474989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:56.474989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:21:56.474989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:21:56.474989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:21:56.475984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:21:56.475984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:56.475984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:56.475984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:56.475984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:56.475984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:56.475984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:56.475984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:56.475984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:56.475984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:56.475984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:56.475984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:56.475984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:56.475984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:56.475984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:56.475984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:56.475984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:56.476970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:21:56.476970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:21:56.476970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:21:56.476970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:56.476970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 525/1034] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:56.482431 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03f44f-39ed-4070-b8ea-ce169502236a None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:56.577271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e30b5-4ea4-491e-93d5-61cde18a31a1 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:56.592543 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03f44f-39ed-4070-b8ea-ce169502236a None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:56.613140 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:56.616448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9861e0-ea7e-4b67-83c3-3338ca1a8989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:56.617123 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9861e0-ea7e-4b67-83c3-3338ca1a8989 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:56.617401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9861e0-ea7e-4b67-83c3-3338ca1a8989 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:56.617676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9861e0-ea7e-4b67-83c3-3338ca1a8989 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:56.618389 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9861e0-ea7e-4b67-83c3-3338ca1a8989 None admin] RBAC: Adding query filter params (type=image) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:56.618699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9861e0-ea7e-4b67-83c3-3338ca1a8989 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:56.620121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9861e0-ea7e-4b67-83c3-3338ca1a8989 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:56.625649 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 510/1035] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:56.637517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9861e0-ea7e-4b67-83c3-3338ca1a8989 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:56.648907 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc8446-ed60-4f2c-b633-5f7696eadcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:56.649934 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc8446-ed60-4f2c-b633-5f7696eadcd7 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:56.650455 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc8446-ed60-4f2c-b633-5f7696eadcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:56.650988 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc8446-ed60-4f2c-b633-5f7696eadcd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:56.652885 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc8446-ed60-4f2c-b633-5f7696eadcd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:56.653460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc8446-ed60-4f2c-b633-5f7696eadcd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:56.656122 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc8446-ed60-4f2c-b633-5f7696eadcd7 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:56.678987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 526/1036] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:56.683662 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc8446-ed60-4f2c-b633-5f7696eadcd7 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:56.731393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:56.796190 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9861e0-ea7e-4b67-83c3-3338ca1a8989 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:56.823286 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc8446-ed60-4f2c-b633-5f7696eadcd7 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:56.831547 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc12147-1862-4bc7-86e0-f58de566fd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:56.832229 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc12147-1862-4bc7-86e0-f58de566fd10 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:56.832536 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc12147-1862-4bc7-86e0-f58de566fd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:56.832772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc12147-1862-4bc7-86e0-f58de566fd10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:56.833572 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc12147-1862-4bc7-86e0-f58de566fd10 None admin] RBAC: Adding query filter params (interface=public, service_id=a7083f37eff0415eb158a07b3632e4bb, region_id=RegionOne) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:56.833883 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc12147-1862-4bc7-86e0-f58de566fd10 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:56.835319 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc12147-1862-4bc7-86e0-f58de566fd10 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:56.841320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 511/1037] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 14 20:21:56 2022] GET /identity/v3/endpoints?service_id=a7083f37eff0415eb158a07b3632e4bb&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:56.842626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12587373-eb58-4846-bd40-de7195e4964a None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:56.842917 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12587373-eb58-4846-bd40-de7195e4964a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:56.843229 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12587373-eb58-4846-bd40-de7195e4964a None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:56.844905 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 512/1038] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:56.847914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12587373-eb58-4846-bd40-de7195e4964a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:56.863125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60ae94-aed0-462f-8d4f-fd3adf29d7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:56.863937 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60ae94-aed0-462f-8d4f-fd3adf29d7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:56.864220 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60ae94-aed0-462f-8d4f-fd3adf29d7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:56.864502 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60ae94-aed0-462f-8d4f-fd3adf29d7b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:56.871899 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60ae94-aed0-462f-8d4f-fd3adf29d7b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:56.873608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60ae94-aed0-462f-8d4f-fd3adf29d7b4 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:56.890776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-8c60ae94-aed0-462f-8d4f-fd3adf29d7b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:56.890776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:56.890776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:56.890776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:56.890776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:56.890776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:56.890776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:56.890776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:56.890776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:56.890776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:56.890776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:56.890776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:56.890776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:21:56.890776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:21:56.890776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:21:56.891958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:21:56.891958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:56.891958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:56.891958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:56.891958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:56.891958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:56.891958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:56.891958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:56.891958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:56.891958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:56.891958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:56.891958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:56.891958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:56.891958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:56.891958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:56.891958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:56.891958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:56.892973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:21:56.892973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:21:56.892973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:21:56.892973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:21:56.892973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:21:56.892973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:21:56.892973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:21:56.892973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:56.892973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 527/1039] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:21:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:21:56.893544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f922e28-b092-4fef-b161-6ed7d309118e None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:56.893810 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f922e28-b092-4fef-b161-6ed7d309118e None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:56.894278 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f922e28-b092-4fef-b161-6ed7d309118e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:56.912420 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-6f922e28-b092-4fef-b161-6ed7d309118e None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:56.913636 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6f922e28-b092-4fef-b161-6ed7d309118e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:56.947520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:56.970459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12587373-eb58-4846-bd40-de7195e4964a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:56.998095 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 528/1040] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:56.998969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f922e28-b092-4fef-b161-6ed7d309118e None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:57.018466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:57.032080 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652f68e-a9f2-4462-a968-a8c780b1080e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:57.032737 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652f68e-a9f2-4462-a968-a8c780b1080e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:57.033026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652f68e-a9f2-4462-a968-a8c780b1080e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:57.033306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652f68e-a9f2-4462-a968-a8c780b1080e None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:57.034001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652f68e-a9f2-4462-a968-a8c780b1080e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:57.035420 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652f68e-a9f2-4462-a968-a8c780b1080e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:57.042700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 513/1041] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 14 20:21:56 2022] GET /identity/v3/services => generated 1957 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:21:57.047018 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652f68e-a9f2-4462-a968-a8c780b1080e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:57.127107 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f922e28-b092-4fef-b161-6ed7d309118e None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:57.167727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7a120-2d44-4921-a8bd-2af47b45820e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:57.168457 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7a120-2d44-4921-a8bd-2af47b45820e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:57.168743 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7a120-2d44-4921-a8bd-2af47b45820e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:57.169019 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7a120-2d44-4921-a8bd-2af47b45820e None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:57.169835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7a120-2d44-4921-a8bd-2af47b45820e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:57.170145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7a120-2d44-4921-a8bd-2af47b45820e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:57.170310 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652f68e-a9f2-4462-a968-a8c780b1080e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:57.171635 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7a120-2d44-4921-a8bd-2af47b45820e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:57.186422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 529/1042] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:21:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:57.193738 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7a120-2d44-4921-a8bd-2af47b45820e None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:57.217926 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f3469-3ec4-474b-b4bf-18586238fb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:57.218801 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f3469-3ec4-474b-b4bf-18586238fb29 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:57.219130 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f3469-3ec4-474b-b4bf-18586238fb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:57.219413 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f3469-3ec4-474b-b4bf-18586238fb29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:57.220707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f3469-3ec4-474b-b4bf-18586238fb29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:57.222185 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f3469-3ec4-474b-b4bf-18586238fb29 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:57.224257 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 514/1043] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:21:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:21:57.230131 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f3469-3ec4-474b-b4bf-18586238fb29 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:57.235361 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:57.306982 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:57.353476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7a120-2d44-4921-a8bd-2af47b45820e None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:57.361585 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f3469-3ec4-474b-b4bf-18586238fb29 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:57.400987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb609d-8179-43a0-a9c5-fa5ad2c19881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:57.402004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb609d-8179-43a0-a9c5-fa5ad2c19881 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:57.402472 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb609d-8179-43a0-a9c5-fa5ad2c19881 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:57.402878 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb609d-8179-43a0-a9c5-fa5ad2c19881 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:57.404388 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb609d-8179-43a0-a9c5-fa5ad2c19881 None admin] RBAC: Adding query filter params (role.id=f932ef767ec247d8ad3823e16f1b0306, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=aa3bb9f1e24a4e9ebd460e97b3736bc0) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:21:57.404880 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb609d-8179-43a0-a9c5-fa5ad2c19881 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:57.407379 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb609d-8179-43a0-a9c5-fa5ad2c19881 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:57.415581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3927f-dc7b-4168-b993-82ddb8534278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:57.416626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3927f-dc7b-4168-b993-82ddb8534278 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:57.427085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3927f-dc7b-4168-b993-82ddb8534278 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:57.427291 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 530/1044] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:21:57 2022] GET /identity/v3/role_assignments?user.id=aa3bb9f1e24a4e9ebd460e97b3736bc0&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=f932ef767ec247d8ad3823e16f1b0306 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:21:57.427374 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3927f-dc7b-4168-b993-82ddb8534278 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:57.427374 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3927f-dc7b-4168-b993-82ddb8534278 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:21:57.427374 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3927f-dc7b-4168-b993-82ddb8534278 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:21:57.460082 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 515/1045] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 14 20:21:57 2022] POST /identity/v3/users => generated 285 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:21:59.484830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a92e7c-4680-44b7-b312-f6d793d1b874 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:59.485433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a92e7c-4680-44b7-b312-f6d793d1b874 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:59.485433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a92e7c-4680-44b7-b312-f6d793d1b874 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:59.486750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 531/1046] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:59.493373 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1a995-af22-4f19-9104-7e346e15a7f6 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:59.493857 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1a995-af22-4f19-9104-7e346e15a7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:59.493935 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1a995-af22-4f19-9104-7e346e15a7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:59.511099 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-b4b1a995-af22-4f19-9104-7e346e15a7f6 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:59.512338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b4b1a995-af22-4f19-9104-7e346e15a7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:59.587430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 516/1047] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:21:59.609432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a92e7c-4680-44b7-b312-f6d793d1b874 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:59.675251 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f0eb7-fb62-49db-a0c1-7abf3cd44ee8 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:59.675518 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f0eb7-fb62-49db-a0c1-7abf3cd44ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:59.675808 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f0eb7-fb62-49db-a0c1-7abf3cd44ee8 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:59.677607 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 517/1048] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:21:59.683188 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a903f9b-4b89-4520-a3c1-661a0b5751a5 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:59.683472 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a903f9b-4b89-4520-a3c1-661a0b5751a5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:59.684395 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a903f9b-4b89-4520-a3c1-661a0b5751a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:59.702033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-0a903f9b-4b89-4520-a3c1-661a0b5751a5 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:59.703287 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0a903f9b-4b89-4520-a3c1-661a0b5751a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:59.741502 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:59.752758 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:21:59.757607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a92e7c-4680-44b7-b312-f6d793d1b874 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:21:59.783410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 518/1049] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 20:21:59.784770 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c8a5c-7880-4f11-b402-569484b880e6 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:59.785100 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c8a5c-7880-4f11-b402-569484b880e6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:59.785339 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c8a5c-7880-4f11-b402-569484b880e6 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:59.786878 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 519/1050] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:21:59.794576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac2b2b-40d8-46dc-892b-49aeadb0ddcd None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:59.795078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac2b2b-40d8-46dc-892b-49aeadb0ddcd None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:59.795459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac2b2b-40d8-46dc-892b-49aeadb0ddcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:59.814634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5db3d-43dc-4feb-8084-29d526fbabcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:21:59.816109 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5db3d-43dc-4feb-8084-29d526fbabcf None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:21:59.816637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5db3d-43dc-4feb-8084-29d526fbabcf None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:21:59.817121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5db3d-43dc-4feb-8084-29d526fbabcf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:21:59.818569 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-11ac2b2b-40d8-46dc-892b-49aeadb0ddcd None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:21:59.819814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-11ac2b2b-40d8-46dc-892b-49aeadb0ddcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:21:59.830686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5db3d-43dc-4feb-8084-29d526fbabcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:21:59.839589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-9db5db3d-43dc-4feb-8084-29d526fbabcf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 20:21:59.839589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:21:59.839589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:21:59.839589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:21:59.839589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:21:59.839589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:21:59.839589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:21:59.839589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:21:59.839589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:21:59.839589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:21:59.839589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:21:59.839589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:21:59.839589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:21:59.839589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:21:59.839589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:21:59.839589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:21:59.840853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:21:59.840853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:21:59.840853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:21:59.840853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:21:59.840853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:21:59.840853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:21:59.840853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:21:59.840853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:21:59.840853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:21:59.840853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:21:59.840853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:21:59.840853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:21:59.840853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:21:59.840853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:21:59.840853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:21:59.840853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:21:59.841980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:21:59.841980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:21:59.841980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:21:59.841980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:21:59.841980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:21:59.841980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:21:59.841980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:21:59.841980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:21:59.841980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:21:59.841980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:21:59.841980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 20:21:59.841980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:21:59.841980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 532/1051] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jul 14 20:21:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:21:59.841980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5db3d-43dc-4feb-8084-29d526fbabcf None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:59.900084 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 520/1052] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:21:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 20:21:59.901914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac2b2b-40d8-46dc-892b-49aeadb0ddcd None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:21:59.959575 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5db3d-43dc-4feb-8084-29d526fbabcf None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:00.008948 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227017f-4288-4a68-afc2-14b01518bdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:00.010086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227017f-4288-4a68-afc2-14b01518bdd8 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:00.010484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227017f-4288-4a68-afc2-14b01518bdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:00.010865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227017f-4288-4a68-afc2-14b01518bdd8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:00.022566 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac2b2b-40d8-46dc-892b-49aeadb0ddcd None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:00.025163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:00.026835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227017f-4288-4a68-afc2-14b01518bdd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:22:00.039282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-7227017f-4288-4a68-afc2-14b01518bdd8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:22:00.039282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:00.039282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:00.039282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:00.039282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:00.039282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:00.039282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:00.039282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:00.039282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:00.039282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:00.039282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:00.039282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:00.039282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:22:00.039282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:22:00.039282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:22:00.039282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:22:00.040407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:22:00.040407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:22:00.040407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:22:00.040407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:22:00.040407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:00.040407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:00.040407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:00.040407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:00.040407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:00.040407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:00.040407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:00.040407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:00.040407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:00.040407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:00.040407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:00.040407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:00.041306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:00.041306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:00.041306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:00.041306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:00.041306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:22:00.041306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:22:00.041306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:22:00.041306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:22:00.041306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:22:00.041306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:22:00.041306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:22:00.041306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:00.041306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 533/1053] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:21:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:22:00.042330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227017f-4288-4a68-afc2-14b01518bdd8 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:00.063400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:00.071723 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793f0fe-1169-4b62-a145-af065af854a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:00.072450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793f0fe-1169-4b62-a145-af065af854a1 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:00.072739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793f0fe-1169-4b62-a145-af065af854a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:00.073019 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793f0fe-1169-4b62-a145-af065af854a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:00.073860 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793f0fe-1169-4b62-a145-af065af854a1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:00.074287 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793f0fe-1169-4b62-a145-af065af854a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:00.075858 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793f0fe-1169-4b62-a145-af065af854a1 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:00.091171 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 521/1054] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 14 20:21:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:00.092209 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793f0fe-1169-4b62-a145-af065af854a1 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:00.175714 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227017f-4288-4a68-afc2-14b01518bdd8 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:00.216546 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbb76e-1062-41b7-a257-841c20fbdd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:00.217341 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbb76e-1062-41b7-a257-841c20fbdd12 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:00.217622 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbb76e-1062-41b7-a257-841c20fbdd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:00.217895 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbb76e-1062-41b7-a257-841c20fbdd12 None admin] PATH_INFO: `/v3/services/image` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:00.218567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bbb76e-1062-41b7-a257-841c20fbdd12 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:00.219959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bbb76e-1062-41b7-a257-841c20fbdd12 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:00.242999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-e7bbb76e-1062-41b7-a257-841c20fbdd12 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 20:22:00.242999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:00.242999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:22:00.242999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:22:00.242999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:22:00.242999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:22:00.242999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:22:00.242999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:22:00.242999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 20:22:00.242999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:00.242999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:22:00.242999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:00.242999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:00.242999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:00.242999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:00.242999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:00.242999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:00.244104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:00.244104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:00.244104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:00.244104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:00.244104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:00.244104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:00.244104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:22:00.244104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:22:00.244104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:22:00.244104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:22:00.244104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:00.244104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:00.244104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:00.244104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:00.244104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:00.244104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:00.244963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:00.244963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:00.244963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:00.244963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:00.244963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:00.244963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:00.244963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:00.244963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:00.244963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:00.244963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:00.244963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:22:00.244963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:22:00.244963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 20:22:00.244963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:00.244963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 534/1055] 10.222.0.52 () {56 vars in 1105 bytes} [Thu Jul 14 20:22:00 2022] GET /identity/v3/services/image => generated 86 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:22:00.245767 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbb76e-1062-41b7-a257-841c20fbdd12 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:00.269269 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793f0fe-1169-4b62-a145-af065af854a1 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:00.314888 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21dbd7-7ddf-49c9-af54-8a8a4fa974ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:00.315585 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21dbd7-7ddf-49c9-af54-8a8a4fa974ab None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:00.317421 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21dbd7-7ddf-49c9-af54-8a8a4fa974ab None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:00.317421 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21dbd7-7ddf-49c9-af54-8a8a4fa974ab None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:00.317421 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21dbd7-7ddf-49c9-af54-8a8a4fa974ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:00.317421 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21dbd7-7ddf-49c9-af54-8a8a4fa974ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:00.318558 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21dbd7-7ddf-49c9-af54-8a8a4fa974ab None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:00.327103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 522/1056] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:22:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:00.328106 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21dbd7-7ddf-49c9-af54-8a8a4fa974ab None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:00.346301 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:00.360499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:00.381649 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbb76e-1062-41b7-a257-841c20fbdd12 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:00.419326 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0c1f3-84c8-4825-9d7e-3607a8c9a93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:00.420118 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0c1f3-84c8-4825-9d7e-3607a8c9a93a None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:00.420400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0c1f3-84c8-4825-9d7e-3607a8c9a93a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:00.420664 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0c1f3-84c8-4825-9d7e-3607a8c9a93a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:00.431726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0c1f3-84c8-4825-9d7e-3607a8c9a93a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:00.433195 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0c1f3-84c8-4825-9d7e-3607a8c9a93a None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:00.440417 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21dbd7-7ddf-49c9-af54-8a8a4fa974ab None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:00.452372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-e4d0c1f3-84c8-4825-9d7e-3607a8c9a93a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:22:00.452372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:00.452372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:00.452372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:00.452372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:00.452372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:00.452372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:00.452372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:00.452372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:00.452372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:00.452372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:00.452372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:00.452372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:22:00.452372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:22:00.452372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:22:00.453350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:22:00.453350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:00.453350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:00.453350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:00.453350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:00.453350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:00.453350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:00.453350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:00.453350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:00.453350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:00.453350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:00.453350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:00.453350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:00.453350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:00.453350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:00.453350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:00.453350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:00.454261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:22:00.454261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:22:00.454261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:22:00.454261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:00.454261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 535/1057] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:22:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:22:00.454536 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0c1f3-84c8-4825-9d7e-3607a8c9a93a None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:00.483422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2304d39-4d36-4a61-a22a-0f73b6b8a21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:00.484096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2304d39-4d36-4a61-a22a-0f73b6b8a21c None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:00.484377 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2304d39-4d36-4a61-a22a-0f73b6b8a21c None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:00.484653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2304d39-4d36-4a61-a22a-0f73b6b8a21c None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:00.485362 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2304d39-4d36-4a61-a22a-0f73b6b8a21c None admin] RBAC: Adding query filter params (name=image) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:00.485669 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2304d39-4d36-4a61-a22a-0f73b6b8a21c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:00.487074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2304d39-4d36-4a61-a22a-0f73b6b8a21c None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:00.492810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 523/1058] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:22:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:00.493531 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2304d39-4d36-4a61-a22a-0f73b6b8a21c None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:00.570123 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0c1f3-84c8-4825-9d7e-3607a8c9a93a None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:00.599431 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:00.601237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:00.608864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2304d39-4d36-4a61-a22a-0f73b6b8a21c None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:00.611135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c160e3-6651-4e7b-990d-803b8e6f9412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:00.612388 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c160e3-6651-4e7b-990d-803b8e6f9412 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:00.612787 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c160e3-6651-4e7b-990d-803b8e6f9412 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:00.613173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c160e3-6651-4e7b-990d-803b8e6f9412 None admin] PATH_INFO: `/v3/users/04c23848696148ae8de5ef826d4a2417` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:00.632492 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c160e3-6651-4e7b-990d-803b8e6f9412 None admin] RBAC: Authorizing `identity:get_user(user_id=04c23848696148ae8de5ef826d4a2417)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:00.634526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c160e3-6651-4e7b-990d-803b8e6f9412 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:00.641535 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 536/1059] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:22:00 2022] GET /identity/v3/users/04c23848696148ae8de5ef826d4a2417 => generated 285 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:00.641535 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c160e3-6651-4e7b-990d-803b8e6f9412 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:00.686620 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898e473-b9f6-4a54-b935-a0f3837de99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:00.687649 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898e473-b9f6-4a54-b935-a0f3837de99b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:00.688060 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898e473-b9f6-4a54-b935-a0f3837de99b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:00.688467 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898e473-b9f6-4a54-b935-a0f3837de99b None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:00.689579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898e473-b9f6-4a54-b935-a0f3837de99b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:00.690075 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898e473-b9f6-4a54-b935-a0f3837de99b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:00.692085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898e473-b9f6-4a54-b935-a0f3837de99b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:00.709683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 524/1060] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:00.710682 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898e473-b9f6-4a54-b935-a0f3837de99b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:00.759404 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c160e3-6651-4e7b-990d-803b8e6f9412 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:00.799615 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9f462-f7d4-40f9-a3cc-93f285523293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:00.800286 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9f462-f7d4-40f9-a3cc-93f285523293 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:00.800568 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9f462-f7d4-40f9-a3cc-93f285523293 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:00.800844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9f462-f7d4-40f9-a3cc-93f285523293 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:00.801555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9f462-f7d4-40f9-a3cc-93f285523293 None admin] RBAC: Adding query filter params (type=image) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:00.801859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9f462-f7d4-40f9-a3cc-93f285523293 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:00.803242 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9f462-f7d4-40f9-a3cc-93f285523293 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:00.809008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 537/1061] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:22:00 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:00.809910 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9f462-f7d4-40f9-a3cc-93f285523293 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:00.838401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898e473-b9f6-4a54-b935-a0f3837de99b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:00.867257 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:00.878512 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842c273-349d-4956-88ab-11bf47d2e9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:00.879205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842c273-349d-4956-88ab-11bf47d2e9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:00.879490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842c273-349d-4956-88ab-11bf47d2e9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:00.879773 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842c273-349d-4956-88ab-11bf47d2e9a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:00.903108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:00.922926 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9f462-f7d4-40f9-a3cc-93f285523293 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:00.957599 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842c273-349d-4956-88ab-11bf47d2e9a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:00.959101 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842c273-349d-4956-88ab-11bf47d2e9a0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:00.960086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea42cf-1f71-4a14-ac4e-1bfec65e8c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:00.960889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea42cf-1f71-4a14-ac4e-1bfec65e8c8d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:00.961177 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea42cf-1f71-4a14-ac4e-1bfec65e8c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:00.961451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea42cf-1f71-4a14-ac4e-1bfec65e8c8d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:00.968943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea42cf-1f71-4a14-ac4e-1bfec65e8c8d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:00.970386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea42cf-1f71-4a14-ac4e-1bfec65e8c8d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:00.987576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-e8ea42cf-1f71-4a14-ac4e-1bfec65e8c8d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 20:22:00.987576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:00.987576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:00.987576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:00.987576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:00.987576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:00.987576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:00.987576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:00.987576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:00.987576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:00.987576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:00.987576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:00.987576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:22:00.987576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:22:00.987576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:22:00.987576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:22:00.988677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:00.988677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:00.988677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:22:00.988677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:22:00.988677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:22:00.988677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:22:00.988677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:00.988677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:00.988677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:00.988677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:00.988677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:00.988677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:00.988677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:00.988677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:00.988677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:00.988677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:00.988677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:00.988677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:00.989833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:00.989833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:00.989833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:22:00.989833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:22:00.989833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:22:00.989833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:22:00.989833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:22:00.989833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:22:00.989833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 20:22:00.989833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:00.989833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 538/1062] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 14 20:22:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:22:00.990439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea42cf-1f71-4a14-ac4e-1bfec65e8c8d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:01.070998 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 525/1063] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:22:00 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:22:01.071893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842c273-349d-4956-88ab-11bf47d2e9a0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:01.108298 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea42cf-1f71-4a14-ac4e-1bfec65e8c8d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:01.118548 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:01.132804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:01.145996 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd84db2-7140-4c4d-8e0c-aad8ba709b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:01.146664 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd84db2-7140-4c4d-8e0c-aad8ba709b01 None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:01.146978 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd84db2-7140-4c4d-8e0c-aad8ba709b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:01.147266 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd84db2-7140-4c4d-8e0c-aad8ba709b01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:01.148015 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd84db2-7140-4c4d-8e0c-aad8ba709b01 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:01.149390 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd84db2-7140-4c4d-8e0c-aad8ba709b01 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:01.168613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 539/1064] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 14 20:22:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:22:01.169449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd84db2-7140-4c4d-8e0c-aad8ba709b01 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:01.184449 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842c273-349d-4956-88ab-11bf47d2e9a0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:01.237684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3884d87-8f5d-45e0-9554-184210d9b0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:01.237684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3884d87-8f5d-45e0-9554-184210d9b0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:01.237684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3884d87-8f5d-45e0-9554-184210d9b0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:01.237684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3884d87-8f5d-45e0-9554-184210d9b0a0 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:01.238139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3884d87-8f5d-45e0-9554-184210d9b0a0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:01.238139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3884d87-8f5d-45e0-9554-184210d9b0a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:01.238870 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3884d87-8f5d-45e0-9554-184210d9b0a0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:01.266565 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 526/1065] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 14 20:22:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:01.267912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3884d87-8f5d-45e0-9554-184210d9b0a0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:01.348138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd84db2-7140-4c4d-8e0c-aad8ba709b01 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:01.394360 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b74cb-0828-4fa4-b348-12bb47221ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:01.395268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b74cb-0828-4fa4-b348-12bb47221ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:01.395557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b74cb-0828-4fa4-b348-12bb47221ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:01.395846 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b74cb-0828-4fa4-b348-12bb47221ee8 None admin] PATH_INFO: `/v3/projects/d2ac10ce063c42c68b908d9043e5c2f9` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:01.397064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b74cb-0828-4fa4-b348-12bb47221ee8 None admin] RBAC: Authorizing `identity:get_project(project_id=d2ac10ce063c42c68b908d9043e5c2f9)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:01.398505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b74cb-0828-4fa4-b348-12bb47221ee8 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:01.402976 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 540/1066] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:22:01 2022] GET /identity/v3/projects/d2ac10ce063c42c68b908d9043e5c2f9 => generated 311 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:01.406479 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b74cb-0828-4fa4-b348-12bb47221ee8 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:01.439467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:01.452030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3884d87-8f5d-45e0-9554-184210d9b0a0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:01.466464 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:01.506638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a7fa3-beda-42b7-8410-2c580c8cb4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:01.507480 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a7fa3-beda-42b7-8410-2c580c8cb4fc None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:01.507767 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a7fa3-beda-42b7-8410-2c580c8cb4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:01.508043 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a7fa3-beda-42b7-8410-2c580c8cb4fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:01.520385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a7fa3-beda-42b7-8410-2c580c8cb4fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:01.521866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a7fa3-beda-42b7-8410-2c580c8cb4fc None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:01.532577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b74cb-0828-4fa4-b348-12bb47221ee8 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:01.541258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-722a7fa3-beda-42b7-8410-2c580c8cb4fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:22:01.541258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:01.541258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:01.541258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:01.541258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:01.541258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:01.541258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:01.541258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:01.541258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:01.541258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:01.541258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:01.541258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:01.541258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:22:01.541258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:22:01.541258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:22:01.542271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:22:01.542271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:01.542271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:01.542271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:01.542271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:01.542271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:01.542271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:01.542271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:01.542271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:01.542271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:01.542271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:01.542271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:01.542271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:01.542271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:01.542271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:01.542271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:01.542271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:01.544110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:22:01.544110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:22:01.544110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:22:01.544110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:22:01.550413 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 527/1067] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:22:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:22:01.551527 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a7fa3-beda-42b7-8410-2c580c8cb4fc None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:01.569858 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e72df9-bec0-49dc-868e-980b8563cb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:01.570525 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e72df9-bec0-49dc-868e-980b8563cb75 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:01.571118 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e72df9-bec0-49dc-868e-980b8563cb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:01.571118 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e72df9-bec0-49dc-868e-980b8563cb75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:01.571921 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e72df9-bec0-49dc-868e-980b8563cb75 None admin] RBAC: Adding query filter params (role.id=5ca112885467422684e134cb0f94865a, scope.project.id=d2ac10ce063c42c68b908d9043e5c2f9, user.id=04c23848696148ae8de5ef826d4a2417) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:01.572227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e72df9-bec0-49dc-868e-980b8563cb75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:01.573661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e72df9-bec0-49dc-868e-980b8563cb75 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:01.580420 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 541/1068] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:22:01 2022] GET /identity/v3/role_assignments?user.id=04c23848696148ae8de5ef826d4a2417&scope.project.id=d2ac10ce063c42c68b908d9043e5c2f9&role.id=5ca112885467422684e134cb0f94865a => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:22:01.709965 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a7fa3-beda-42b7-8410-2c580c8cb4fc None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:01.763718 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3044b4c-a252-4c48-ac43-276f444c10e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:01.764387 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3044b4c-a252-4c48-ac43-276f444c10e7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:01.764774 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3044b4c-a252-4c48-ac43-276f444c10e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:01.765056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3044b4c-a252-4c48-ac43-276f444c10e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:01.765788 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3044b4c-a252-4c48-ac43-276f444c10e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:01.766100 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3044b4c-a252-4c48-ac43-276f444c10e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:01.768014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3044b4c-a252-4c48-ac43-276f444c10e7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:01.784346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 528/1069] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:01.792271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e72df9-bec0-49dc-868e-980b8563cb75 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:01.917550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:01.925995 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e72df9-bec0-49dc-868e-980b8563cb75 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:01.961529 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f406cf-7d8b-4eab-8ec9-816480238b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:01.962220 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f406cf-7d8b-4eab-8ec9-816480238b2d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:01.962499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f406cf-7d8b-4eab-8ec9-816480238b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:01.962769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f406cf-7d8b-4eab-8ec9-816480238b2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:01.970723 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f406cf-7d8b-4eab-8ec9-816480238b2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:01.972717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f406cf-7d8b-4eab-8ec9-816480238b2d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:01.989465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-16f406cf-7d8b-4eab-8ec9-816480238b2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:22:01.989465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:01.989465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:01.989465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:01.989465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:01.989465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:01.989465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:01.989465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:01.989465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:01.989465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:01.989465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:01.989465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:01.989465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:22:01.989465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:22:01.989465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:22:01.991530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:22:01.991530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:01.991530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:01.991530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:01.991530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:01.991530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:01.991530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:01.991530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:01.991530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:01.991530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:01.991530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:01.991530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:01.991530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:01.991530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:01.991530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:01.991530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:01.991530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:01.993488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:22:01.993488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:22:01.993488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:22:01.993488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:22:01.993488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:22:01.993488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:22:01.993488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:22:01.993488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:01.995505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 542/1070] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:22:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:22:01.997516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3044b4c-a252-4c48-ac43-276f444c10e7 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:02.031930 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:02.149521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3044b4c-a252-4c48-ac43-276f444c10e7 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:02.183911 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd0b74-2f81-426d-a93e-9854a7d6350a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:02.184581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd0b74-2f81-426d-a93e-9854a7d6350a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:02.184863 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd0b74-2f81-426d-a93e-9854a7d6350a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:02.185139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd0b74-2f81-426d-a93e-9854a7d6350a None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:02.186486 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd0b74-2f81-426d-a93e-9854a7d6350a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:02.186813 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd0b74-2f81-426d-a93e-9854a7d6350a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:02.188394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd0b74-2f81-426d-a93e-9854a7d6350a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:02.206673 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 529/1071] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:22:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:02.207938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f406cf-7d8b-4eab-8ec9-816480238b2d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:02.315083 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f406cf-7d8b-4eab-8ec9-816480238b2d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:02.350999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c8c94-eeb8-426c-bf1c-2b1ee134dc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:02.351919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c8c94-eeb8-426c-bf1c-2b1ee134dc2b None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:02.352200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c8c94-eeb8-426c-bf1c-2b1ee134dc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:02.352477 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c8c94-eeb8-426c-bf1c-2b1ee134dc2b None admin] PATH_INFO: `/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/aa3bb9f1e24a4e9ebd460e97b3736bc0/roles/f932ef767ec247d8ad3823e16f1b0306` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:02.354861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c8c94-eeb8-426c-bf1c-2b1ee134dc2b None admin] RBAC: Authorizing `identity:create_grant(project_id=f33c8ac5154f46a89d1a0dc11fb58cf4, user_id=aa3bb9f1e24a4e9ebd460e97b3736bc0, role_id=f932ef767ec247d8ad3823e16f1b0306)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:02.356431 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c8c94-eeb8-426c-bf1c-2b1ee134dc2b None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:02.378968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 543/1072] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:22:02 2022] PUT /identity/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/aa3bb9f1e24a4e9ebd460e97b3736bc0/roles/f932ef767ec247d8ad3823e16f1b0306 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:22:03.315265 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ec106-4121-4f6e-864a-cc5a4ef8a1c1 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:03.315966 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ec106-4121-4f6e-864a-cc5a4ef8a1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:03.315966 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ec106-4121-4f6e-864a-cc5a4ef8a1c1 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:03.317381 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 530/1073] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:22:03.322467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239106d4-09e7-4503-92c7-b2cf221d27cb None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:03.322886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239106d4-09e7-4503-92c7-b2cf221d27cb None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:03.323116 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239106d4-09e7-4503-92c7-b2cf221d27cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:03.342450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-239106d4-09e7-4503-92c7-b2cf221d27cb None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:03.344134 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-239106d4-09e7-4503-92c7-b2cf221d27cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:03.361797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:03.432270 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 544/1074] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:22:03.454120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ec106-4121-4f6e-864a-cc5a4ef8a1c1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:03.542196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:03.569902 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ec106-4121-4f6e-864a-cc5a4ef8a1c1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:03.607599 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d09f4a-90e1-47c9-b8cb-66878e33c67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:03.608253 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d09f4a-90e1-47c9-b8cb-66878e33c67e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:03.608537 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d09f4a-90e1-47c9-b8cb-66878e33c67e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:03.608814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d09f4a-90e1-47c9-b8cb-66878e33c67e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:03.698820 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d09f4a-90e1-47c9-b8cb-66878e33c67e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:03.700277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d09f4a-90e1-47c9-b8cb-66878e33c67e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:03.782310 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:03.807518 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 531/1075] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:22:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:22:03.819020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239106d4-09e7-4503-92c7-b2cf221d27cb None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:03.921460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239106d4-09e7-4503-92c7-b2cf221d27cb None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:03.956818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d806cc4-243d-49e3-ac5b-3b15b07b59e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:03.957424 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d806cc4-243d-49e3-ac5b-3b15b07b59e2 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:03.957699 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d806cc4-243d-49e3-ac5b-3b15b07b59e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:03.957967 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d806cc4-243d-49e3-ac5b-3b15b07b59e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:03.968838 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d806cc4-243d-49e3-ac5b-3b15b07b59e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:03.970400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d806cc4-243d-49e3-ac5b-3b15b07b59e2 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:03.977264 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:03.989720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-1d806cc4-243d-49e3-ac5b-3b15b07b59e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:22:03.989720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:03.989720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:03.989720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:03.989720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:03.989720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:03.989720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:03.989720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:03.989720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:03.989720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:03.989720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:03.989720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:03.989720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:22:03.989720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:22:03.989720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:22:03.990769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:22:03.990769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:03.990769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:03.990769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:03.990769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:03.990769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:03.990769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:03.990769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:03.990769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:03.990769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:03.990769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:03.990769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:03.990769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:03.990769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:03.990769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:03.990769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:03.990769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:03.991720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:22:03.991720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:22:03.991720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:22:03.991720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:03.992547 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 545/1076] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:22:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:22:03.997671 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d09f4a-90e1-47c9-b8cb-66878e33c67e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:04.012492 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e6be0-66c6-4c64-87f0-15589f234e6a None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:04.012830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e6be0-66c6-4c64-87f0-15589f234e6a None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:04.013076 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e6be0-66c6-4c64-87f0-15589f234e6a None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:04.014745 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 546/1077] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:22:04.019823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6a088-9251-4e4b-8dc6-6e56664f147b None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:04.020111 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6a088-9251-4e4b-8dc6-6e56664f147b None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:04.020387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6a088-9251-4e4b-8dc6-6e56664f147b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:04.037842 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-40f6a088-9251-4e4b-8dc6-6e56664f147b None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:04.039083 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-40f6a088-9251-4e4b-8dc6-6e56664f147b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:04.109545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d09f4a-90e1-47c9-b8cb-66878e33c67e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:04.114978 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 547/1078] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 20:22:04.134260 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6a088-9251-4e4b-8dc6-6e56664f147b None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:04.152546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91246d-8535-45fc-bf38-9bd1c600fb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:04.153196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91246d-8535-45fc-bf38-9bd1c600fb05 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:04.153478 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91246d-8535-45fc-bf38-9bd1c600fb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:04.153751 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91246d-8535-45fc-bf38-9bd1c600fb05 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:04.154470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91246d-8535-45fc-bf38-9bd1c600fb05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:04.154777 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91246d-8535-45fc-bf38-9bd1c600fb05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:04.156178 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91246d-8535-45fc-bf38-9bd1c600fb05 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:04.179450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 532/1079] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:04.261844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6a088-9251-4e4b-8dc6-6e56664f147b None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:04.303633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:04.323260 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212734e-b248-46dd-a120-be730e84b4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:04.324562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212734e-b248-46dd-a120-be730e84b4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:04.324973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212734e-b248-46dd-a120-be730e84b4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:04.325384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212734e-b248-46dd-a120-be730e84b4b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:04.341988 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212734e-b248-46dd-a120-be730e84b4b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:22:04.354262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-f212734e-b248-46dd-a120-be730e84b4b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:22:04.354262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:04.354262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:04.354262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:04.354262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:04.354262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:04.354262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:04.354262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:04.354262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:04.354262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:04.354262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:04.354262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:04.354262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:22:04.354262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:22:04.354262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:22:04.354262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:22:04.355991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:22:04.355991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:22:04.355991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:22:04.355991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:22:04.355991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:04.355991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:04.355991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:04.355991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:04.355991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:04.355991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:04.355991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:04.355991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:04.355991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:04.355991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:04.355991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:04.355991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:04.356864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:04.356864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:04.356864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:04.356864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:04.356864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:22:04.356864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:22:04.356864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:22:04.356864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:22:04.356864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:22:04.356864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:22:04.356864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:22:04.356864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:04.356864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 548/1080] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:22:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:22:04.364041 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91246d-8535-45fc-bf38-9bd1c600fb05 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:04.411677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:04.475790 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91246d-8535-45fc-bf38-9bd1c600fb05 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:04.512414 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914402d-cb98-4632-a758-ce47e1376e37 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:04.512839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914402d-cb98-4632-a758-ce47e1376e37 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:04.513132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914402d-cb98-4632-a758-ce47e1376e37 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:04.514515 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 549/1081] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:22:04.519973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0916a4a-a779-4e81-a649-92ff2ea83b35 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:04.520264 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0916a4a-a779-4e81-a649-92ff2ea83b35 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:04.520532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0916a4a-a779-4e81-a649-92ff2ea83b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:04.525193 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559c390-a0c8-4620-b875-b424bd612af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:04.526058 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559c390-a0c8-4620-b875-b424bd612af4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:04.527285 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559c390-a0c8-4620-b875-b424bd612af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:04.527585 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559c390-a0c8-4620-b875-b424bd612af4 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:04.528357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559c390-a0c8-4620-b875-b424bd612af4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:04.528668 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559c390-a0c8-4620-b875-b424bd612af4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:04.530056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559c390-a0c8-4620-b875-b424bd612af4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:04.537761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-c0916a4a-a779-4e81-a649-92ff2ea83b35 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:04.538893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 533/1082] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:22:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:04.539138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-c0916a4a-a779-4e81-a649-92ff2ea83b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:04.545445 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559c390-a0c8-4620-b875-b424bd612af4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:04.609340 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 550/1083] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 20:22:04.633104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0916a4a-a779-4e81-a649-92ff2ea83b35 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:04.647000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:04.650809 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559c390-a0c8-4620-b875-b424bd612af4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:04.687915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9abf6-fc17-4c0a-b64f-1d055a2cca00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:04.688804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9abf6-fc17-4c0a-b64f-1d055a2cca00 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:04.689088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9abf6-fc17-4c0a-b64f-1d055a2cca00 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:04.689364 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9abf6-fc17-4c0a-b64f-1d055a2cca00 None admin] PATH_INFO: `/v3/users/04c23848696148ae8de5ef826d4a2417` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:04.690604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd9abf6-fc17-4c0a-b64f-1d055a2cca00 None admin] RBAC: Authorizing `identity:get_user(user_id=04c23848696148ae8de5ef826d4a2417)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:04.692098 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd9abf6-fc17-4c0a-b64f-1d055a2cca00 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:04.693779 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 534/1084] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:22:04 2022] GET /identity/v3/users/04c23848696148ae8de5ef826d4a2417 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:22:04.699571 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9abf6-fc17-4c0a-b64f-1d055a2cca00 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:04.743606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0916a4a-a779-4e81-a649-92ff2ea83b35 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:04.760918 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:04.780175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de38939-6154-4b2e-b86d-d4b11f5664bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:04.780988 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de38939-6154-4b2e-b86d-d4b11f5664bc None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:04.781267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de38939-6154-4b2e-b86d-d4b11f5664bc None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:04.781536 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de38939-6154-4b2e-b86d-d4b11f5664bc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:04.792073 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de38939-6154-4b2e-b86d-d4b11f5664bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:22:04.800952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-0de38939-6154-4b2e-b86d-d4b11f5664bc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 20:22:04.800952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:04.800952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:04.800952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:04.800952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:04.800952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:04.800952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:04.800952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:04.800952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:04.800952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:04.800952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:04.800952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:04.800952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:22:04.800952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:22:04.800952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:22:04.800952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:22:04.802725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:22:04.802725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:22:04.802725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:22:04.802725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:22:04.802725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:04.802725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:04.802725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:04.802725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:04.802725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:04.802725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:04.802725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:04.802725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:04.802725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:04.802725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:04.802725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:04.802725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:04.804643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:04.804643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:04.804643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:04.804643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:04.804643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:22:04.804643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:22:04.804643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:22:04.804643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:22:04.804643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:22:04.804643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:22:04.804643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 20:22:04.804643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:04.804643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 551/1085] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jul 14 20:22:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:22:04.808216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de38939-6154-4b2e-b86d-d4b11f5664bc None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:04.811956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9abf6-fc17-4c0a-b64f-1d055a2cca00 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:04.852403 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342c31e-6b29-4d32-9a30-db63c32b0d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:04.853264 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342c31e-6b29-4d32-9a30-db63c32b0d23 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:04.853543 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342c31e-6b29-4d32-9a30-db63c32b0d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:04.853815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342c31e-6b29-4d32-9a30-db63c32b0d23 None admin] PATH_INFO: `/v3/projects/d2ac10ce063c42c68b908d9043e5c2f9` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:04.855017 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342c31e-6b29-4d32-9a30-db63c32b0d23 None admin] RBAC: Authorizing `identity:get_project(project_id=d2ac10ce063c42c68b908d9043e5c2f9)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:04.857037 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342c31e-6b29-4d32-9a30-db63c32b0d23 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:04.858850 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 535/1086] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:22:04 2022] GET /identity/v3/projects/d2ac10ce063c42c68b908d9043e5c2f9 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:22:04.864881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342c31e-6b29-4d32-9a30-db63c32b0d23 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:04.899723 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:04.913748 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de38939-6154-4b2e-b86d-d4b11f5664bc None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:04.948801 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f43bd-314f-4da8-a360-bcf384588092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:04.950003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f43bd-314f-4da8-a360-bcf384588092 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:04.950280 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f43bd-314f-4da8-a360-bcf384588092 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:04.950629 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f43bd-314f-4da8-a360-bcf384588092 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:04.951335 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f43bd-314f-4da8-a360-bcf384588092 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:04.951639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f43bd-314f-4da8-a360-bcf384588092 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:04.952984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f43bd-314f-4da8-a360-bcf384588092 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:04.960924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 552/1087] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 14 20:22:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:22:04.967372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f43bd-314f-4da8-a360-bcf384588092 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:04.970295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342c31e-6b29-4d32-9a30-db63c32b0d23 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:05.007826 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b8cf5-16be-4880-9ade-a34adf83e1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:05.008530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:05.009068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b8cf5-16be-4880-9ade-a34adf83e1f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:05.009360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b8cf5-16be-4880-9ade-a34adf83e1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:05.009647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b8cf5-16be-4880-9ade-a34adf83e1f3 None admin] PATH_INFO: `/v3/projects/d2ac10ce063c42c68b908d9043e5c2f9/users/04c23848696148ae8de5ef826d4a2417/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:05.012920 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b8cf5-16be-4880-9ade-a34adf83e1f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d2ac10ce063c42c68b908d9043e5c2f9, user_id=04c23848696148ae8de5ef826d4a2417, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:05.014572 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b8cf5-16be-4880-9ade-a34adf83e1f3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:05.033511 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 536/1088] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:22:04 2022] PUT /identity/v3/projects/d2ac10ce063c42c68b908d9043e5c2f9/users/04c23848696148ae8de5ef826d4a2417/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:22:05.073899 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f43bd-314f-4da8-a360-bcf384588092 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:05.114112 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e223ef4-8791-4be4-9040-37dd099a7da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:05.115025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e223ef4-8791-4be4-9040-37dd099a7da3 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:05.115318 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e223ef4-8791-4be4-9040-37dd099a7da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:05.115728 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e223ef4-8791-4be4-9040-37dd099a7da3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:05.128241 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e223ef4-8791-4be4-9040-37dd099a7da3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:05.129732 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e223ef4-8791-4be4-9040-37dd099a7da3 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:05.149939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-3e223ef4-8791-4be4-9040-37dd099a7da3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:22:05.149939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:05.149939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:05.149939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:05.149939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:05.149939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:05.149939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:05.149939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:05.149939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:05.149939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:05.149939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:05.149939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:05.149939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:22:05.149939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:22:05.149939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:22:05.151229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:22:05.151229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:05.151229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:05.151229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:05.151229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:05.151229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:05.151229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:05.151229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:05.151229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:05.151229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:05.151229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:05.151229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:05.151229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:05.151229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:05.151229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:05.151229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:05.151229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:05.153147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:22:05.153147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:22:05.153147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:22:05.153147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:05.153147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 553/1089] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:22:05.163944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b8cf5-16be-4880-9ade-a34adf83e1f3 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:05.272738 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:05.280457 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b8cf5-16be-4880-9ade-a34adf83e1f3 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:05.316453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deadd50-ac57-49a1-8db4-9e902d4a18fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:05.317126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deadd50-ac57-49a1-8db4-9e902d4a18fa None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:05.317426 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deadd50-ac57-49a1-8db4-9e902d4a18fa None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:05.317689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deadd50-ac57-49a1-8db4-9e902d4a18fa None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:05.318416 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deadd50-ac57-49a1-8db4-9e902d4a18fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:05.318727 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deadd50-ac57-49a1-8db4-9e902d4a18fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:05.320142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deadd50-ac57-49a1-8db4-9e902d4a18fa None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:05.332481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 537/1090] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:05.338181 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e223ef4-8791-4be4-9040-37dd099a7da3 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:05.446530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:05.465193 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e223ef4-8791-4be4-9040-37dd099a7da3 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:05.500238 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5488e-c849-475e-8de5-a0175ac6c4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:05.501041 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5488e-c849-475e-8de5-a0175ac6c4ff None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:05.501324 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5488e-c849-475e-8de5-a0175ac6c4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:05.501598 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5488e-c849-475e-8de5-a0175ac6c4ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:05.509566 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5488e-c849-475e-8de5-a0175ac6c4ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:05.511195 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5488e-c849-475e-8de5-a0175ac6c4ff None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:05.526011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-1ea5488e-c849-475e-8de5-a0175ac6c4ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 20:22:05.526011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:05.526011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:05.526011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:05.526011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:05.526011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:05.526011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:05.526011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:05.526011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:05.526011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:05.526011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:05.526011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:05.526011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:22:05.526011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:22:05.526011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:22:05.526011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:22:05.528537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:05.528537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:05.528537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:22:05.528537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:22:05.528537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:22:05.528537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:22:05.528537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:05.528537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:05.528537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:05.528537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:05.528537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:05.528537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:05.528537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:05.528537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:05.528537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:05.528537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:05.528537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:05.528537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:05.530473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:05.530473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:05.530473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:22:05.530473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:22:05.530473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:22:05.530473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:22:05.530473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:22:05.530473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:22:05.530473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 20:22:05.530473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:05.531782 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 554/1091] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 14 20:22:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:22:05.533664 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deadd50-ac57-49a1-8db4-9e902d4a18fa None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:05.636183 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deadd50-ac57-49a1-8db4-9e902d4a18fa None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:05.672548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5790a9-27af-4cac-be10-0e77fe60b749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:05.673224 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5790a9-27af-4cac-be10-0e77fe60b749 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:05.673510 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5790a9-27af-4cac-be10-0e77fe60b749 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:05.673781 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5790a9-27af-4cac-be10-0e77fe60b749 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:05.674623 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5790a9-27af-4cac-be10-0e77fe60b749 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:05.674950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5790a9-27af-4cac-be10-0e77fe60b749 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:05.676441 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5790a9-27af-4cac-be10-0e77fe60b749 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:05.697845 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 538/1092] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 14 20:22:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:05.704130 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5488e-c849-475e-8de5-a0175ac6c4ff None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:05.814787 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5488e-c849-475e-8de5-a0175ac6c4ff None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:05.850912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568c6bf-7d23-4b7c-92ba-198dd8fcc6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:05.852049 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568c6bf-7d23-4b7c-92ba-198dd8fcc6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:05.852554 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568c6bf-7d23-4b7c-92ba-198dd8fcc6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:05.853053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568c6bf-7d23-4b7c-92ba-198dd8fcc6c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:05.877869 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:05.925852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568c6bf-7d23-4b7c-92ba-198dd8fcc6c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:05.927338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568c6bf-7d23-4b7c-92ba-198dd8fcc6c4 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:06.033448 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 555/1093] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:22:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:22:06.039365 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5790a9-27af-4cac-be10-0e77fe60b749 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:06.055023 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:06.141229 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5790a9-27af-4cac-be10-0e77fe60b749 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:06.177243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638af512-c892-4c30-9bd6-2921dbe7b3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:06.178685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638af512-c892-4c30-9bd6-2921dbe7b3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:06.178685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638af512-c892-4c30-9bd6-2921dbe7b3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:06.178860 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638af512-c892-4c30-9bd6-2921dbe7b3a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:06.190504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638af512-c892-4c30-9bd6-2921dbe7b3a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:06.192010 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638af512-c892-4c30-9bd6-2921dbe7b3a9 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:06.210550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-638af512-c892-4c30-9bd6-2921dbe7b3a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:22:06.210550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:06.210550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:06.210550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:06.210550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:06.210550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:06.210550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:06.210550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:06.210550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:06.210550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:06.210550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:06.210550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:06.210550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:22:06.210550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:22:06.210550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:22:06.211546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:22:06.211546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:06.211546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:06.211546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:06.211546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:06.211546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:06.211546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:06.211546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:06.211546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:06.211546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:06.211546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:06.211546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:06.211546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:06.211546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:06.211546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:06.211546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:06.211546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:06.212462 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:22:06.212462 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:22:06.212462 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:22:06.212462 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:22:06.212462 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 539/1094] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:22:06.217625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568c6bf-7d23-4b7c-92ba-198dd8fcc6c4 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:06.275413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:06.324704 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568c6bf-7d23-4b7c-92ba-198dd8fcc6c4 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:06.361037 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03291495-6f5f-4ee7-83fb-81406e654b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:06.361710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03291495-6f5f-4ee7-83fb-81406e654b66 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:06.361991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03291495-6f5f-4ee7-83fb-81406e654b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:06.362265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03291495-6f5f-4ee7-83fb-81406e654b66 None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:06.363026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03291495-6f5f-4ee7-83fb-81406e654b66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:06.363363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03291495-6f5f-4ee7-83fb-81406e654b66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:06.364735 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03291495-6f5f-4ee7-83fb-81406e654b66 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:06.381335 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 556/1095] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:06.382524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638af512-c892-4c30-9bd6-2921dbe7b3a9 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:06.449271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:06.484326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638af512-c892-4c30-9bd6-2921dbe7b3a9 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:06.518575 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a49353-e66b-4398-a2cf-f6eb21b06116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:06.519663 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a49353-e66b-4398-a2cf-f6eb21b06116 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:06.519945 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a49353-e66b-4398-a2cf-f6eb21b06116 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:06.520217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a49353-e66b-4398-a2cf-f6eb21b06116 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:06.527206 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a49353-e66b-4398-a2cf-f6eb21b06116 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:06.528646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a49353-e66b-4398-a2cf-f6eb21b06116 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:06.542124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-49a49353-e66b-4398-a2cf-f6eb21b06116 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:22:06.542124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:06.542124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:06.542124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:06.542124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:06.542124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:06.542124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:06.542124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:06.542124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:06.542124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:06.542124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:06.542124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:06.542124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:22:06.542124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:22:06.542124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:22:06.544176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:22:06.544176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:06.544176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:06.544176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:06.544176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:06.544176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:06.544176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:06.544176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:06.544176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:06.544176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:06.544176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:06.544176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:06.544176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:06.544176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:06.544176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:06.544176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:06.544176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:06.545320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:22:06.545320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:22:06.545320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:22:06.545320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:22:06.545320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:22:06.545320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:22:06.545320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:22:06.545320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:22:06.547223 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 540/1096] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:22:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:22:06.549802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03291495-6f5f-4ee7-83fb-81406e654b66 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:06.653454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03291495-6f5f-4ee7-83fb-81406e654b66 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:06.666323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:06.691425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4521310-8371-4d2e-ba6f-db11296fb804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:06.692126 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4521310-8371-4d2e-ba6f-db11296fb804 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:06.692428 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4521310-8371-4d2e-ba6f-db11296fb804 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:06.692690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4521310-8371-4d2e-ba6f-db11296fb804 None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:06.693546 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4521310-8371-4d2e-ba6f-db11296fb804 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:06.693802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4521310-8371-4d2e-ba6f-db11296fb804 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:06.695273 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4521310-8371-4d2e-ba6f-db11296fb804 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:06.708373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 557/1097] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:22:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:06.715899 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a49353-e66b-4398-a2cf-f6eb21b06116 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:06.823799 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a49353-e66b-4398-a2cf-f6eb21b06116 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:06.849300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:06.858916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bbc35-8e73-4b6e-a14c-11fc8dac1ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:06.859598 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bbc35-8e73-4b6e-a14c-11fc8dac1ada None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:06.859876 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bbc35-8e73-4b6e-a14c-11fc8dac1ada None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:06.860151 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bbc35-8e73-4b6e-a14c-11fc8dac1ada None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:06.860981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bbc35-8e73-4b6e-a14c-11fc8dac1ada None admin] RBAC: Adding query filter params (role.id=f932ef767ec247d8ad3823e16f1b0306, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=aa3bb9f1e24a4e9ebd460e97b3736bc0) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:06.861287 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bbc35-8e73-4b6e-a14c-11fc8dac1ada None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:06.862706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bbc35-8e73-4b6e-a14c-11fc8dac1ada None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:06.868635 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 541/1098] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:22:06 2022] GET /identity/v3/role_assignments?user.id=aa3bb9f1e24a4e9ebd460e97b3736bc0&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=f932ef767ec247d8ad3823e16f1b0306 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:07.088056 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97577890-5abf-4e93-8ea8-ad0cf782b26e None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:07.088344 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97577890-5abf-4e93-8ea8-ad0cf782b26e None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:07.088622 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97577890-5abf-4e93-8ea8-ad0cf782b26e None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:07.090517 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 558/1099] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:22:07.096125 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53c82c-ec86-4511-be93-7dfda8f32a98 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:07.096430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53c82c-ec86-4511-be93-7dfda8f32a98 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:07.096714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53c82c-ec86-4511-be93-7dfda8f32a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:07.114656 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1a53c82c-ec86-4511-be93-7dfda8f32a98 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:07.115927 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1a53c82c-ec86-4511-be93-7dfda8f32a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:07.189038 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 542/1100] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:22:07.212546 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97577890-5abf-4e93-8ea8-ad0cf782b26e None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:07.322824 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97577890-5abf-4e93-8ea8-ad0cf782b26e None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:07.359110 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346bbda-0f0c-4bbc-8ec1-932261170d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:07.359837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346bbda-0f0c-4bbc-8ec1-932261170d81 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:07.360115 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346bbda-0f0c-4bbc-8ec1-932261170d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:07.360388 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346bbda-0f0c-4bbc-8ec1-932261170d81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:07.371691 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346bbda-0f0c-4bbc-8ec1-932261170d81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:22:07.379445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-c346bbda-0f0c-4bbc-8ec1-932261170d81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:22:07.379445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:07.379445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:07.379445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:07.379445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:07.379445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:07.379445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:07.379445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:07.379445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:07.379445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:07.379445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:07.379445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:07.379445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:22:07.379445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:22:07.379445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:22:07.379445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:22:07.380611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:22:07.380611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:22:07.380611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:22:07.380611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:22:07.380611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:07.380611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:07.380611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:07.380611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:07.380611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:07.380611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:07.380611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:07.380611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:07.380611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:07.380611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:07.380611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:07.380611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:07.381727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:07.381727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:07.381727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:07.381727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:07.381727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:22:07.381727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:22:07.381727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:22:07.381727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:22:07.381727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:22:07.381727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:22:07.381727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:22:07.381727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:07.381727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 559/1101] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:22:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:22:07.387052 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53c82c-ec86-4511-be93-7dfda8f32a98 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:07.491967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53c82c-ec86-4511-be93-7dfda8f32a98 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:07.513685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:07.528860 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514e026-1c45-42f6-8b9e-22cc77468aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:07.529528 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514e026-1c45-42f6-8b9e-22cc77468aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:07.529813 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514e026-1c45-42f6-8b9e-22cc77468aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:07.530086 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514e026-1c45-42f6-8b9e-22cc77468aa9 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:07.530829 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514e026-1c45-42f6-8b9e-22cc77468aa9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:07.531167 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514e026-1c45-42f6-8b9e-22cc77468aa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:07.532531 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514e026-1c45-42f6-8b9e-22cc77468aa9 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:07.541358 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 543/1102] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:22:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:07.547535 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346bbda-0f0c-4bbc-8ec1-932261170d81 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:07.581733 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:07.660599 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346bbda-0f0c-4bbc-8ec1-932261170d81 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:07.697554 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dda1c8-0845-4c47-bd43-4c6be9a0a335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:07.698453 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dda1c8-0845-4c47-bd43-4c6be9a0a335 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:07.698732 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dda1c8-0845-4c47-bd43-4c6be9a0a335 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:07.699041 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dda1c8-0845-4c47-bd43-4c6be9a0a335 None admin] PATH_INFO: `/v3/users/04c23848696148ae8de5ef826d4a2417` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:07.700292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dda1c8-0845-4c47-bd43-4c6be9a0a335 None admin] RBAC: Authorizing `identity:get_user(user_id=04c23848696148ae8de5ef826d4a2417)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:07.701732 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dda1c8-0845-4c47-bd43-4c6be9a0a335 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:07.703462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 560/1103] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:22:07 2022] GET /identity/v3/users/04c23848696148ae8de5ef826d4a2417 => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:07.709476 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514e026-1c45-42f6-8b9e-22cc77468aa9 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:07.813809 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514e026-1c45-42f6-8b9e-22cc77468aa9 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:07.846463 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f697c-6a4a-48ae-844d-9b02aa4ecd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:07.847131 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f697c-6a4a-48ae-844d-9b02aa4ecd74 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:07.847419 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f697c-6a4a-48ae-844d-9b02aa4ecd74 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:07.847695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f697c-6a4a-48ae-844d-9b02aa4ecd74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:07.895186 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:07.914023 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f697c-6a4a-48ae-844d-9b02aa4ecd74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:07.915509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f697c-6a4a-48ae-844d-9b02aa4ecd74 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:08.012544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 544/1104] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:22:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:22:08.018430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dda1c8-0845-4c47-bd43-4c6be9a0a335 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:08.112950 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:08.115841 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dda1c8-0845-4c47-bd43-4c6be9a0a335 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:08.149114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdddf14-2f9f-4302-a391-6e197079c778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:08.149907 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdddf14-2f9f-4302-a391-6e197079c778 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:08.150188 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdddf14-2f9f-4302-a391-6e197079c778 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:08.150464 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdddf14-2f9f-4302-a391-6e197079c778 None admin] PATH_INFO: `/v3/projects/d2ac10ce063c42c68b908d9043e5c2f9` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:08.151653 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdddf14-2f9f-4302-a391-6e197079c778 None admin] RBAC: Authorizing `identity:get_project(project_id=d2ac10ce063c42c68b908d9043e5c2f9)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:08.153678 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdddf14-2f9f-4302-a391-6e197079c778 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:08.155409 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 561/1105] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:22:08 2022] GET /identity/v3/projects/d2ac10ce063c42c68b908d9043e5c2f9 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:08.161553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f697c-6a4a-48ae-844d-9b02aa4ecd74 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:08.240483 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:08.258646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f697c-6a4a-48ae-844d-9b02aa4ecd74 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:08.293446 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aaf11e-35ea-4cd7-bc47-5643c3ee90ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:08.294133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aaf11e-35ea-4cd7-bc47-5643c3ee90ba None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:08.294422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aaf11e-35ea-4cd7-bc47-5643c3ee90ba None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:08.294702 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aaf11e-35ea-4cd7-bc47-5643c3ee90ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:08.295566 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aaf11e-35ea-4cd7-bc47-5643c3ee90ba None admin] RBAC: Adding query filter params (role.id=5ca112885467422684e134cb0f94865a, scope.project.id=d2ac10ce063c42c68b908d9043e5c2f9, user.id=04c23848696148ae8de5ef826d4a2417) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:08.295907 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aaf11e-35ea-4cd7-bc47-5643c3ee90ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:08.297338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aaf11e-35ea-4cd7-bc47-5643c3ee90ba None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:08.303206 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 545/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:22:08 2022] GET /identity/v3/role_assignments?user.id=04c23848696148ae8de5ef826d4a2417&scope.project.id=d2ac10ce063c42c68b908d9043e5c2f9&role.id=5ca112885467422684e134cb0f94865a => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:10.365477 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ccf5a2-480d-4384-9575-55a8855ff7a5 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:10.366064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ccf5a2-480d-4384-9575-55a8855ff7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:10.366064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ccf5a2-480d-4384-9575-55a8855ff7a5 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:10.367436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 562/1107] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:22:10.373129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31397c9b-9392-4064-87c5-aef6b640c74f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:10.373533 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31397c9b-9392-4064-87c5-aef6b640c74f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:10.373690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31397c9b-9392-4064-87c5-aef6b640c74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:10.390127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-31397c9b-9392-4064-87c5-aef6b640c74f None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:10.391362 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-31397c9b-9392-4064-87c5-aef6b640c74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:10.456763 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 546/1108] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:22:10.483488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ccf5a2-480d-4384-9575-55a8855ff7a5 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:10.591180 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ccf5a2-480d-4384-9575-55a8855ff7a5 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:10.624344 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9b9f2-7cc1-4b0b-9593-f4b5f12b2745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:10.625062 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9b9f2-7cc1-4b0b-9593-f4b5f12b2745 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:10.625348 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9b9f2-7cc1-4b0b-9593-f4b5f12b2745 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:10.625634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9b9f2-7cc1-4b0b-9593-f4b5f12b2745 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:10.626850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9b9f2-7cc1-4b0b-9593-f4b5f12b2745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:10.628321 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9b9f2-7cc1-4b0b-9593-f4b5f12b2745 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:10.629964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 563/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:22:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:10.635727 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31397c9b-9392-4064-87c5-aef6b640c74f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:10.715364 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:10.739235 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31397c9b-9392-4064-87c5-aef6b640c74f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:10.775052 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899c0ac-88c8-4f4a-bdcd-730ed46d45e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:10.775737 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899c0ac-88c8-4f4a-bdcd-730ed46d45e6 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:10.776027 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899c0ac-88c8-4f4a-bdcd-730ed46d45e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:10.776304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899c0ac-88c8-4f4a-bdcd-730ed46d45e6 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:10.777566 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899c0ac-88c8-4f4a-bdcd-730ed46d45e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:10.779025 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899c0ac-88c8-4f4a-bdcd-730ed46d45e6 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:10.798867 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 547/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 14 20:22:10 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:22:12.846068 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656ba70-2ece-44d0-971e-91deb66df9e1 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:12.846614 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656ba70-2ece-44d0-971e-91deb66df9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:12.846614 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656ba70-2ece-44d0-971e-91deb66df9e1 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:12.848022 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 564/1111] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:22:12.853261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656c412-2712-4bd2-af09-6ee439e45349 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:12.853659 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656c412-2712-4bd2-af09-6ee439e45349 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:12.853825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656c412-2712-4bd2-af09-6ee439e45349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:12.870243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8656c412-2712-4bd2-af09-6ee439e45349 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:12.871977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8656c412-2712-4bd2-af09-6ee439e45349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:12.939546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 548/1112] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:22:12.962772 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656ba70-2ece-44d0-971e-91deb66df9e1 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:12.999006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:13.064587 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656ba70-2ece-44d0-971e-91deb66df9e1 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:13.105925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334f030-2d70-4286-b653-21c6190e75ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:13.106775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334f030-2d70-4286-b653-21c6190e75ce None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:13.107092 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334f030-2d70-4286-b653-21c6190e75ce None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:13.107372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334f030-2d70-4286-b653-21c6190e75ce None admin] PATH_INFO: `/v3/roles/6b2eb3e4cefd4a2db4f18e04a158600a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:13.108842 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334f030-2d70-4286-b653-21c6190e75ce None admin] RBAC: Authorizing `identity:get_role(role_id=6b2eb3e4cefd4a2db4f18e04a158600a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:13.110216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334f030-2d70-4286-b653-21c6190e75ce None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:13.111658 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 565/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:22:12 2022] GET /identity/v3/roles/6b2eb3e4cefd4a2db4f18e04a158600a => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:13.118425 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656c412-2712-4bd2-af09-6ee439e45349 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:13.186301 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:13.224944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656c412-2712-4bd2-af09-6ee439e45349 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:13.257257 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa753b-c201-47e0-8ac6-71e7ad3f7196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:13.258122 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa753b-c201-47e0-8ac6-71e7ad3f7196 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:13.258403 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa753b-c201-47e0-8ac6-71e7ad3f7196 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:13.258677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa753b-c201-47e0-8ac6-71e7ad3f7196 None admin] PATH_INFO: `/v3/users/94c63fa90d374832ac38064eae274366` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:13.272304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa753b-c201-47e0-8ac6-71e7ad3f7196 None admin] RBAC: Authorizing `identity:get_user(user_id=94c63fa90d374832ac38064eae274366)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:13.273732 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa753b-c201-47e0-8ac6-71e7ad3f7196 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:13.275425 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 549/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:22:13 2022] GET /identity/v3/users/94c63fa90d374832ac38064eae274366 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:13.281771 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334f030-2d70-4286-b653-21c6190e75ce None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:13.378089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334f030-2d70-4286-b653-21c6190e75ce None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:13.380370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:13.412176 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb8909-60dc-496a-ba5c-7ff4c6cc5054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:13.412826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb8909-60dc-496a-ba5c-7ff4c6cc5054 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:13.413114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb8909-60dc-496a-ba5c-7ff4c6cc5054 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:13.413394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb8909-60dc-496a-ba5c-7ff4c6cc5054 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:13.479054 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb8909-60dc-496a-ba5c-7ff4c6cc5054 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:13.480455 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb8909-60dc-496a-ba5c-7ff4c6cc5054 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:13.575645 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 566/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:22:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:22:13.581602 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa753b-c201-47e0-8ac6-71e7ad3f7196 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:13.680746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa753b-c201-47e0-8ac6-71e7ad3f7196 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:13.701993 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:13.716246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca49b70-03c0-4705-93a6-420a3348126f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:13.717112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca49b70-03c0-4705-93a6-420a3348126f None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:13.717404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca49b70-03c0-4705-93a6-420a3348126f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:13.717672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca49b70-03c0-4705-93a6-420a3348126f None admin] PATH_INFO: `/v3/projects/d2ac10ce063c42c68b908d9043e5c2f9` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:13.718849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca49b70-03c0-4705-93a6-420a3348126f None admin] RBAC: Authorizing `identity:get_project(project_id=d2ac10ce063c42c68b908d9043e5c2f9)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:13.720326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca49b70-03c0-4705-93a6-420a3348126f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:13.722048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 550/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:22:13 2022] GET /identity/v3/projects/d2ac10ce063c42c68b908d9043e5c2f9 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:13.728418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb8909-60dc-496a-ba5c-7ff4c6cc5054 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:13.731732 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:13.825133 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb8909-60dc-496a-ba5c-7ff4c6cc5054 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:13.858410 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f0697-717c-451a-9567-5fc4161d5f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:13.859090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f0697-717c-451a-9567-5fc4161d5f6f None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:13.859364 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f0697-717c-451a-9567-5fc4161d5f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:13.859640 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f0697-717c-451a-9567-5fc4161d5f6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:13.860473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f0697-717c-451a-9567-5fc4161d5f6f None admin] RBAC: Adding query filter params (role.id=6b2eb3e4cefd4a2db4f18e04a158600a, scope.project.id=d2ac10ce063c42c68b908d9043e5c2f9, user.id=94c63fa90d374832ac38064eae274366) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:13.860779 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f0697-717c-451a-9567-5fc4161d5f6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:13.862202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f0697-717c-451a-9567-5fc4161d5f6f None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:13.867897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 567/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:22:13 2022] GET /identity/v3/role_assignments?user.id=94c63fa90d374832ac38064eae274366&scope.project.id=d2ac10ce063c42c68b908d9043e5c2f9&role.id=6b2eb3e4cefd4a2db4f18e04a158600a => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:15.831075 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefec385-bb63-4e79-bb88-9b43dc2ba459 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:15.831620 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefec385-bb63-4e79-bb88-9b43dc2ba459 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:15.831620 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefec385-bb63-4e79-bb88-9b43dc2ba459 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:15.833008 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 551/1118] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:22:15.838190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88a881-034d-4dba-848a-abeb57aadd55 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:15.838625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88a881-034d-4dba-848a-abeb57aadd55 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:15.838801 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88a881-034d-4dba-848a-abeb57aadd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:15.855174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-7b88a881-034d-4dba-848a-abeb57aadd55 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:15.856353 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-7b88a881-034d-4dba-848a-abeb57aadd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:15.916708 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:15.922555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 568/1119] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:22:15.944922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefec385-bb63-4e79-bb88-9b43dc2ba459 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:16.045648 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefec385-bb63-4e79-bb88-9b43dc2ba459 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:16.078812 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18361d4c-313a-4408-979a-fccd4f426f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:16.079683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18361d4c-313a-4408-979a-fccd4f426f4b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:16.079962 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18361d4c-313a-4408-979a-fccd4f426f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:16.080233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18361d4c-313a-4408-979a-fccd4f426f4b None admin] PATH_INFO: `/v3/roles/6b2eb3e4cefd4a2db4f18e04a158600a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:16.081682 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18361d4c-313a-4408-979a-fccd4f426f4b None admin] RBAC: Authorizing `identity:get_role(role_id=6b2eb3e4cefd4a2db4f18e04a158600a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:16.083079 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18361d4c-313a-4408-979a-fccd4f426f4b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:16.084449 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 552/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:22:15 2022] GET /identity/v3/roles/6b2eb3e4cefd4a2db4f18e04a158600a => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:16.090247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88a881-034d-4dba-848a-abeb57aadd55 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:16.190235 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88a881-034d-4dba-848a-abeb57aadd55 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:16.222690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacc619-acdb-4655-b52e-0a97f67eae8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:16.223574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacc619-acdb-4655-b52e-0a97f67eae8a None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:16.223860 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacc619-acdb-4655-b52e-0a97f67eae8a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:16.224135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacc619-acdb-4655-b52e-0a97f67eae8a None admin] PATH_INFO: `/v3/users/94c63fa90d374832ac38064eae274366` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:16.225299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacc619-acdb-4655-b52e-0a97f67eae8a None admin] RBAC: Authorizing `identity:get_user(user_id=94c63fa90d374832ac38064eae274366)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:16.226720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacc619-acdb-4655-b52e-0a97f67eae8a None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:16.228503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 569/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:22:16 2022] GET /identity/v3/users/94c63fa90d374832ac38064eae274366 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:16.235256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18361d4c-313a-4408-979a-fccd4f426f4b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:16.290463 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:16.341930 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18361d4c-313a-4408-979a-fccd4f426f4b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:16.379288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083d5ef-b380-40a0-a8e3-51e5c533fc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:16.380141 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083d5ef-b380-40a0-a8e3-51e5c533fc02 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:16.380418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083d5ef-b380-40a0-a8e3-51e5c533fc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:16.380687 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083d5ef-b380-40a0-a8e3-51e5c533fc02 None admin] PATH_INFO: `/v3/projects/d2ac10ce063c42c68b908d9043e5c2f9` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:16.381855 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083d5ef-b380-40a0-a8e3-51e5c533fc02 None admin] RBAC: Authorizing `identity:get_project(project_id=d2ac10ce063c42c68b908d9043e5c2f9)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:16.383311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083d5ef-b380-40a0-a8e3-51e5c533fc02 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:16.385006 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 553/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:22:16 2022] GET /identity/v3/projects/d2ac10ce063c42c68b908d9043e5c2f9 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:16.391169 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacc619-acdb-4655-b52e-0a97f67eae8a None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:16.455726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:16.489958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacc619-acdb-4655-b52e-0a97f67eae8a None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:16.525016 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4032e-c086-48fe-8750-ed4261acbc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:16.525902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4032e-c086-48fe-8750-ed4261acbc61 None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:16.526182 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4032e-c086-48fe-8750-ed4261acbc61 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:16.526456 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4032e-c086-48fe-8750-ed4261acbc61 None admin] PATH_INFO: `/v3/projects/d2ac10ce063c42c68b908d9043e5c2f9/users/94c63fa90d374832ac38064eae274366/roles/6b2eb3e4cefd4a2db4f18e04a158600a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:16.528609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4032e-c086-48fe-8750-ed4261acbc61 None admin] RBAC: Authorizing `identity:create_grant(project_id=d2ac10ce063c42c68b908d9043e5c2f9, user_id=94c63fa90d374832ac38064eae274366, role_id=6b2eb3e4cefd4a2db4f18e04a158600a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:16.530043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4032e-c086-48fe-8750-ed4261acbc61 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:16.546151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 570/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:22:16 2022] PUT /identity/v3/projects/d2ac10ce063c42c68b908d9043e5c2f9/users/94c63fa90d374832ac38064eae274366/roles/6b2eb3e4cefd4a2db4f18e04a158600a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:22:18.575776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8ba07-c60d-4261-94a0-86b59299a2b1 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:18.576377 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8ba07-c60d-4261-94a0-86b59299a2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:18.576377 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8ba07-c60d-4261-94a0-86b59299a2b1 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:18.577695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 554/1124] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:22:18.583239 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8a50f-8ea7-4a45-b291-a87720047fcd None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:18.584152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8a50f-8ea7-4a45-b291-a87720047fcd None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:18.584152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8a50f-8ea7-4a45-b291-a87720047fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:18.601612 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-fdb8a50f-8ea7-4a45-b291-a87720047fcd None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:18.602807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-fdb8a50f-8ea7-4a45-b291-a87720047fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:18.670010 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 571/1125] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:22:18.693333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8ba07-c60d-4261-94a0-86b59299a2b1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:18.795668 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8ba07-c60d-4261-94a0-86b59299a2b1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:18.818631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:18.830140 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed415beb-aed2-435c-b40c-260701b904be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:18.830921 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed415beb-aed2-435c-b40c-260701b904be None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:18.831230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed415beb-aed2-435c-b40c-260701b904be None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:18.831502 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed415beb-aed2-435c-b40c-260701b904be None admin] PATH_INFO: `/v3/roles/6b2eb3e4cefd4a2db4f18e04a158600a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:18.833000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed415beb-aed2-435c-b40c-260701b904be None admin] RBAC: Authorizing `identity:get_role(role_id=6b2eb3e4cefd4a2db4f18e04a158600a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:18.834358 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed415beb-aed2-435c-b40c-260701b904be None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:18.835750 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 555/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:22:18 2022] GET /identity/v3/roles/6b2eb3e4cefd4a2db4f18e04a158600a => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:18.842594 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8a50f-8ea7-4a45-b291-a87720047fcd None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:18.899182 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:18.947125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8a50f-8ea7-4a45-b291-a87720047fcd None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:18.983835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9862931-934b-41f2-b52c-a3a1c6a21643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:18.984733 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9862931-934b-41f2-b52c-a3a1c6a21643 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:18.985016 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9862931-934b-41f2-b52c-a3a1c6a21643 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:18.985294 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9862931-934b-41f2-b52c-a3a1c6a21643 None admin] PATH_INFO: `/v3/users/94c63fa90d374832ac38064eae274366` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:18.986586 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9862931-934b-41f2-b52c-a3a1c6a21643 None admin] RBAC: Authorizing `identity:get_user(user_id=94c63fa90d374832ac38064eae274366)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:18.988087 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9862931-934b-41f2-b52c-a3a1c6a21643 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:18.989824 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 572/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:22:18 2022] GET /identity/v3/users/94c63fa90d374832ac38064eae274366 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:18.996414 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed415beb-aed2-435c-b40c-260701b904be None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:19.099426 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed415beb-aed2-435c-b40c-260701b904be None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:19.131955 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af11fd8-de75-4105-a9d0-0914def8e94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:19.132590 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af11fd8-de75-4105-a9d0-0914def8e94e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:19.132869 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af11fd8-de75-4105-a9d0-0914def8e94e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:19.133135 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af11fd8-de75-4105-a9d0-0914def8e94e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:19.190366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:19.200366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af11fd8-de75-4105-a9d0-0914def8e94e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:19.201807 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af11fd8-de75-4105-a9d0-0914def8e94e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:19.302532 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 556/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:22:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:22:19.309090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9862931-934b-41f2-b52c-a3a1c6a21643 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:19.406317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9862931-934b-41f2-b52c-a3a1c6a21643 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:19.429364 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:19.441155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a014901-4c0b-4d12-9fc6-9a993725a233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:19.442014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a014901-4c0b-4d12-9fc6-9a993725a233 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:19.442299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a014901-4c0b-4d12-9fc6-9a993725a233 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:19.442580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a014901-4c0b-4d12-9fc6-9a993725a233 None admin] PATH_INFO: `/v3/projects/d2ac10ce063c42c68b908d9043e5c2f9` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:19.443788 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a014901-4c0b-4d12-9fc6-9a993725a233 None admin] RBAC: Authorizing `identity:get_project(project_id=d2ac10ce063c42c68b908d9043e5c2f9)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:19.445215 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a014901-4c0b-4d12-9fc6-9a993725a233 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:19.446919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 573/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:22:19 2022] GET /identity/v3/projects/d2ac10ce063c42c68b908d9043e5c2f9 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:19.453155 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af11fd8-de75-4105-a9d0-0914def8e94e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:19.542753 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:19.551835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af11fd8-de75-4105-a9d0-0914def8e94e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:19.585196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165c850-de52-43df-b07d-ad6f73e7d03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:19.585852 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165c850-de52-43df-b07d-ad6f73e7d03a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:19.586120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165c850-de52-43df-b07d-ad6f73e7d03a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:19.586398 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165c850-de52-43df-b07d-ad6f73e7d03a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:19.587257 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165c850-de52-43df-b07d-ad6f73e7d03a None admin] RBAC: Adding query filter params (role.id=6b2eb3e4cefd4a2db4f18e04a158600a, scope.project.id=d2ac10ce063c42c68b908d9043e5c2f9, user.id=94c63fa90d374832ac38064eae274366) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:19.587573 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165c850-de52-43df-b07d-ad6f73e7d03a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:19.589004 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165c850-de52-43df-b07d-ad6f73e7d03a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:19.595391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 557/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:22:19 2022] GET /identity/v3/role_assignments?user.id=94c63fa90d374832ac38064eae274366&scope.project.id=d2ac10ce063c42c68b908d9043e5c2f9&role.id=6b2eb3e4cefd4a2db4f18e04a158600a => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:21.650391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e55c9-10bc-41d0-86bc-1f9a4b2221ac None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:21.650924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e55c9-10bc-41d0-86bc-1f9a4b2221ac None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:21.650924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e55c9-10bc-41d0-86bc-1f9a4b2221ac None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:21.652398 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 574/1131] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:22:21.657845 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9020b37-8f8d-4693-a63d-13deedd0f711 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:21.658245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9020b37-8f8d-4693-a63d-13deedd0f711 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:21.658394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9020b37-8f8d-4693-a63d-13deedd0f711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:21.675285 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a9020b37-8f8d-4693-a63d-13deedd0f711 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:21.676490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a9020b37-8f8d-4693-a63d-13deedd0f711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:21.742888 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 558/1132] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:22:21.766121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e55c9-10bc-41d0-86bc-1f9a4b2221ac None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:21.870611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e55c9-10bc-41d0-86bc-1f9a4b2221ac None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:21.904733 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac4a3c-79f7-4f9a-a6a9-70fe2715f79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:21.905440 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac4a3c-79f7-4f9a-a6a9-70fe2715f79a None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:21.905725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac4a3c-79f7-4f9a-a6a9-70fe2715f79a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:21.906000 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac4a3c-79f7-4f9a-a6a9-70fe2715f79a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:21.907194 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac4a3c-79f7-4f9a-a6a9-70fe2715f79a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:21.908622 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac4a3c-79f7-4f9a-a6a9-70fe2715f79a None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:21.910252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 575/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:22:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:21.916382 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9020b37-8f8d-4693-a63d-13deedd0f711 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:21.998250 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:22.012743 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9020b37-8f8d-4693-a63d-13deedd0f711 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:22.046250 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c58330-67e8-4cad-8f3c-5735ee0d921f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:22.046905 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c58330-67e8-4cad-8f3c-5735ee0d921f None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:22.047217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c58330-67e8-4cad-8f3c-5735ee0d921f None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:22.047489 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c58330-67e8-4cad-8f3c-5735ee0d921f None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:22.048197 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c58330-67e8-4cad-8f3c-5735ee0d921f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:22.049615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c58330-67e8-4cad-8f3c-5735ee0d921f None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:22.074813 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 559/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 14 20:22:21 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:22:24.128979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11ffa7-ac6f-4d59-9e07-1520bc51ae22 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:24.129539 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11ffa7-ac6f-4d59-9e07-1520bc51ae22 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:24.129539 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11ffa7-ac6f-4d59-9e07-1520bc51ae22 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:24.130898 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 576/1135] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:22:24.136085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5492c8b-c9e5-401f-8bf7-58d4e48a3172 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:24.136467 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5492c8b-c9e5-401f-8bf7-58d4e48a3172 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:24.136642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5492c8b-c9e5-401f-8bf7-58d4e48a3172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:24.155735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d5492c8b-c9e5-401f-8bf7-58d4e48a3172 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:24.156941 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d5492c8b-c9e5-401f-8bf7-58d4e48a3172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:24.221847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 560/1136] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 20:22:24.246278 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11ffa7-ac6f-4d59-9e07-1520bc51ae22 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:24.307201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:24.347953 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11ffa7-ac6f-4d59-9e07-1520bc51ae22 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:24.381099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18c8d9-9011-4ab0-9755-c489cff499ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:24.381802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18c8d9-9011-4ab0-9755-c489cff499ca None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:24.382080 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18c8d9-9011-4ab0-9755-c489cff499ca None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:24.382351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18c8d9-9011-4ab0-9755-c489cff499ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:24.383545 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18c8d9-9011-4ab0-9755-c489cff499ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:24.384962 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18c8d9-9011-4ab0-9755-c489cff499ca None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:24.386951 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 577/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:22:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:24.392953 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5492c8b-c9e5-401f-8bf7-58d4e48a3172 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:24.456869 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:24.489540 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5492c8b-c9e5-401f-8bf7-58d4e48a3172 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:24.522305 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6a633-1747-44e3-8e01-5c71b31d33e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:24.522986 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6a633-1747-44e3-8e01-5c71b31d33e3 None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:24.523269 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6a633-1747-44e3-8e01-5c71b31d33e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:24.523536 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6a633-1747-44e3-8e01-5c71b31d33e3 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:24.524246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be6a633-1747-44e3-8e01-5c71b31d33e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:24.525663 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be6a633-1747-44e3-8e01-5c71b31d33e3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:24.545297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 561/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 14 20:22:24 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:22:26.582555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266500f-5910-46b3-9125-55e2607891ec None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:26.583242 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266500f-5910-46b3-9125-55e2607891ec None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:26.583242 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266500f-5910-46b3-9125-55e2607891ec None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:26.584508 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 578/1139] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:22:26.589742 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2a76f-e012-413d-a343-81b9a869a33b None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:26.590152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2a76f-e012-413d-a343-81b9a869a33b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:26.590298 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2a76f-e012-413d-a343-81b9a869a33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:26.606779 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d2e2a76f-e012-413d-a343-81b9a869a33b None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:26.607981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d2e2a76f-e012-413d-a343-81b9a869a33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:26.672485 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 562/1140] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:22:26.696282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266500f-5910-46b3-9125-55e2607891ec None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:26.795123 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266500f-5910-46b3-9125-55e2607891ec None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:26.820626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:26.828620 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e4f2c-fdeb-4745-813a-e7f59eba590e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:26.829407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e4f2c-fdeb-4745-813a-e7f59eba590e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:26.829688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e4f2c-fdeb-4745-813a-e7f59eba590e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:26.829966 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e4f2c-fdeb-4745-813a-e7f59eba590e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:26.840104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e4f2c-fdeb-4745-813a-e7f59eba590e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:22:26.847555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-ef9e4f2c-fdeb-4745-813a-e7f59eba590e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:22:26.847555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:26.847555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:26.847555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:26.847555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:26.847555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:26.847555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:26.847555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:26.847555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:26.847555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:26.847555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:26.847555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:26.847555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:22:26.847555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:22:26.847555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:22:26.847555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:22:26.848717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:22:26.848717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:22:26.848717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:22:26.848717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:22:26.848717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:26.848717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:26.848717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:26.848717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:26.848717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:26.848717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:26.848717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:26.848717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:26.848717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:26.848717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:26.848717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:26.848717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:26.849818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:26.849818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:26.849818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:26.849818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:26.849818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:22:26.849818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:22:26.849818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:22:26.849818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:22:26.849818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:22:26.849818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:22:26.849818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:22:26.849818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:26.849818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 579/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:22:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:22:26.855683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2a76f-e012-413d-a343-81b9a869a33b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:26.909166 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:26.953243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2a76f-e012-413d-a343-81b9a869a33b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:26.985917 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ba32c-b85b-47d9-ac31-85a911b1502b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:26.986567 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ba32c-b85b-47d9-ac31-85a911b1502b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:26.986849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ba32c-b85b-47d9-ac31-85a911b1502b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:26.987155 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ba32c-b85b-47d9-ac31-85a911b1502b None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:26.987898 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ba32c-b85b-47d9-ac31-85a911b1502b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:26.988207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ba32c-b85b-47d9-ac31-85a911b1502b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:26.989568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ba32c-b85b-47d9-ac31-85a911b1502b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:26.998027 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 563/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:22:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:27.003804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e4f2c-fdeb-4745-813a-e7f59eba590e None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:27.100600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e4f2c-fdeb-4745-813a-e7f59eba590e None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:27.132466 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcac82-00f9-4d9c-8089-3678e7f7d172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:27.133324 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcac82-00f9-4d9c-8089-3678e7f7d172 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:27.133604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcac82-00f9-4d9c-8089-3678e7f7d172 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:27.133876 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcac82-00f9-4d9c-8089-3678e7f7d172 None admin] PATH_INFO: `/v3/users/86d28122b6f4460eb4262976ead44b78` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:27.146645 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcac82-00f9-4d9c-8089-3678e7f7d172 None admin] RBAC: Authorizing `identity:get_user(user_id=86d28122b6f4460eb4262976ead44b78)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:27.148105 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcac82-00f9-4d9c-8089-3678e7f7d172 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:27.149712 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 580/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:22:27 2022] GET /identity/v3/users/86d28122b6f4460eb4262976ead44b78 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:27.156072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ba32c-b85b-47d9-ac31-85a911b1502b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:27.256433 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ba32c-b85b-47d9-ac31-85a911b1502b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:27.271946 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:27.289773 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef0936-9739-447c-90d1-90868a166919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:27.290414 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef0936-9739-447c-90d1-90868a166919 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:27.290698 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef0936-9739-447c-90d1-90868a166919 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:27.290999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef0936-9739-447c-90d1-90868a166919 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:27.360830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef0936-9739-447c-90d1-90868a166919 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:27.362184 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef0936-9739-447c-90d1-90868a166919 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:27.455858 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 564/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:22:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:22:27.461756 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcac82-00f9-4d9c-8089-3678e7f7d172 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:27.507663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:27.559306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcac82-00f9-4d9c-8089-3678e7f7d172 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:27.592244 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1518b-bc24-4fee-8ba3-ba24b3a91e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:27.593100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1518b-bc24-4fee-8ba3-ba24b3a91e43 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:27.593380 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1518b-bc24-4fee-8ba3-ba24b3a91e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:27.593652 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1518b-bc24-4fee-8ba3-ba24b3a91e43 None admin] PATH_INFO: `/v3/projects/bdc190df4add446fa8da2d6b2d2edfeb` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:27.594820 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a1518b-bc24-4fee-8ba3-ba24b3a91e43 None admin] RBAC: Authorizing `identity:get_project(project_id=bdc190df4add446fa8da2d6b2d2edfeb)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:27.596259 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a1518b-bc24-4fee-8ba3-ba24b3a91e43 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:27.597942 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 581/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:22:27 2022] GET /identity/v3/projects/bdc190df4add446fa8da2d6b2d2edfeb => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:27.604189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef0936-9739-447c-90d1-90868a166919 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:27.621061 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:27.702866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef0936-9739-447c-90d1-90868a166919 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:27.735958 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a4473-ec88-4e12-a5a2-dba2de663519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:27.736593 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a4473-ec88-4e12-a5a2-dba2de663519 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:27.736872 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a4473-ec88-4e12-a5a2-dba2de663519 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:27.737141 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a4473-ec88-4e12-a5a2-dba2de663519 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:27.737957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a4473-ec88-4e12-a5a2-dba2de663519 None admin] RBAC: Adding query filter params (role.id=5ca112885467422684e134cb0f94865a, scope.project.id=bdc190df4add446fa8da2d6b2d2edfeb, user.id=86d28122b6f4460eb4262976ead44b78) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:27.738264 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a4473-ec88-4e12-a5a2-dba2de663519 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:27.739710 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a4473-ec88-4e12-a5a2-dba2de663519 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:27.744963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 565/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:22:27 2022] GET /identity/v3/role_assignments?user.id=86d28122b6f4460eb4262976ead44b78&scope.project.id=bdc190df4add446fa8da2d6b2d2edfeb&role.id=5ca112885467422684e134cb0f94865a => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:22:29.706548 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752ccaf-a4d1-49d3-b416-b1a44633dcab None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:29.707071 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752ccaf-a4d1-49d3-b416-b1a44633dcab None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:29.707187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752ccaf-a4d1-49d3-b416-b1a44633dcab None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:29.708500 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 582/1147] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:22:29.713783 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c0b6c-9b0a-4b89-bc23-4715f68b4556 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:29.714204 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c0b6c-9b0a-4b89-bc23-4715f68b4556 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:29.714346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c0b6c-9b0a-4b89-bc23-4715f68b4556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:29.731227 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-569c0b6c-9b0a-4b89-bc23-4715f68b4556 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:29.732416 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-569c0b6c-9b0a-4b89-bc23-4715f68b4556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:29.797409 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 566/1148] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:22:29.820546 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752ccaf-a4d1-49d3-b416-b1a44633dcab None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:29.923419 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752ccaf-a4d1-49d3-b416-b1a44633dcab None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:29.937397 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:29.956739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6f081-909c-45b9-bd19-28de6fc5ad89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:29.957519 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6f081-909c-45b9-bd19-28de6fc5ad89 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:29.957801 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6f081-909c-45b9-bd19-28de6fc5ad89 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:29.958075 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6f081-909c-45b9-bd19-28de6fc5ad89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:29.967892 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece6f081-909c-45b9-bd19-28de6fc5ad89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:22:29.975441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-ece6f081-909c-45b9-bd19-28de6fc5ad89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:22:29.975441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:29.975441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:29.975441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:29.975441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:29.975441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:29.975441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:29.975441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:29.975441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:29.975441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:29.975441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:29.975441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:29.975441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:22:29.975441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:22:29.975441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:22:29.975441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:22:29.976362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:22:29.976362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:22:29.976362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:22:29.976362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:22:29.976362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:29.976362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:29.976362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:29.976362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:29.976362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:29.976362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:29.976362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:29.976362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:29.976362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:29.976362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:29.976362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:29.976362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:29.977213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:29.977213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:29.977213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:29.977213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:29.977213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:22:29.977213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:22:29.977213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:22:29.977213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:22:29.977213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:22:29.977213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:22:29.977213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:22:29.977213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:29.977213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 583/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:22:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:22:29.983381 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c0b6c-9b0a-4b89-bc23-4715f68b4556 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:29.994681 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:30.078012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c0b6c-9b0a-4b89-bc23-4715f68b4556 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:30.110869 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23f509-4786-48fe-b876-461a5f5d51c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:30.111548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23f509-4786-48fe-b876-461a5f5d51c3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:30.111827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23f509-4786-48fe-b876-461a5f5d51c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:30.112100 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23f509-4786-48fe-b876-461a5f5d51c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:30.112834 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23f509-4786-48fe-b876-461a5f5d51c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:30.113139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23f509-4786-48fe-b876-461a5f5d51c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:30.114496 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23f509-4786-48fe-b876-461a5f5d51c3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:30.122274 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 567/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:22:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:30.128137 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6f081-909c-45b9-bd19-28de6fc5ad89 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:30.225457 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6f081-909c-45b9-bd19-28de6fc5ad89 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:30.258113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7341f4-34c2-4f37-a042-ea78ee4c03d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:30.259013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7341f4-34c2-4f37-a042-ea78ee4c03d3 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:30.259298 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7341f4-34c2-4f37-a042-ea78ee4c03d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:30.259572 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7341f4-34c2-4f37-a042-ea78ee4c03d3 None admin] PATH_INFO: `/v3/users/86d28122b6f4460eb4262976ead44b78` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:30.260747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7341f4-34c2-4f37-a042-ea78ee4c03d3 None admin] RBAC: Authorizing `identity:get_user(user_id=86d28122b6f4460eb4262976ead44b78)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:30.262212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7341f4-34c2-4f37-a042-ea78ee4c03d3 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:30.263881 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 584/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:22:30 2022] GET /identity/v3/users/86d28122b6f4460eb4262976ead44b78 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:22:30.269910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23f509-4786-48fe-b876-461a5f5d51c3 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:30.341200 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:30.373981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23f509-4786-48fe-b876-461a5f5d51c3 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:30.408376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17777e7-9a3d-42f0-a01e-e0f347aa4432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:30.409824 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17777e7-9a3d-42f0-a01e-e0f347aa4432 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:30.410128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17777e7-9a3d-42f0-a01e-e0f347aa4432 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:30.410518 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17777e7-9a3d-42f0-a01e-e0f347aa4432 None admin] PATH_INFO: `/v3/projects/bdc190df4add446fa8da2d6b2d2edfeb` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:30.411760 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17777e7-9a3d-42f0-a01e-e0f347aa4432 None admin] RBAC: Authorizing `identity:get_project(project_id=bdc190df4add446fa8da2d6b2d2edfeb)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:30.413195 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17777e7-9a3d-42f0-a01e-e0f347aa4432 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:30.414914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 568/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:22:30 2022] GET /identity/v3/projects/bdc190df4add446fa8da2d6b2d2edfeb => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:30.421371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7341f4-34c2-4f37-a042-ea78ee4c03d3 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:30.463593 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:30.519616 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7341f4-34c2-4f37-a042-ea78ee4c03d3 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:30.551862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec3ef3-104f-4ec3-bd82-de359c90858a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:30.552739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec3ef3-104f-4ec3-bd82-de359c90858a None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:30.553017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec3ef3-104f-4ec3-bd82-de359c90858a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:30.553291 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec3ef3-104f-4ec3-bd82-de359c90858a None admin] PATH_INFO: `/v3/projects/bdc190df4add446fa8da2d6b2d2edfeb/users/86d28122b6f4460eb4262976ead44b78/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:30.555245 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec3ef3-104f-4ec3-bd82-de359c90858a None admin] RBAC: Authorizing `identity:create_grant(project_id=bdc190df4add446fa8da2d6b2d2edfeb, user_id=86d28122b6f4460eb4262976ead44b78, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:30.556697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec3ef3-104f-4ec3-bd82-de359c90858a None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:30.569209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 585/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:22:30 2022] PUT /identity/v3/projects/bdc190df4add446fa8da2d6b2d2edfeb/users/86d28122b6f4460eb4262976ead44b78/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:22:32.547869 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a2f10-52ca-4aff-b127-349f75867a81 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:32.548432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a2f10-52ca-4aff-b127-349f75867a81 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:32.548432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a2f10-52ca-4aff-b127-349f75867a81 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:32.549774 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 569/1154] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:22:32.554840 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd991661-d904-46a5-b486-9b5d00d76711 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:32.555287 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd991661-d904-46a5-b486-9b5d00d76711 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:32.555403 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd991661-d904-46a5-b486-9b5d00d76711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:32.572339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-bd991661-d904-46a5-b486-9b5d00d76711 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:32.574151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-bd991661-d904-46a5-b486-9b5d00d76711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:32.639085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 586/1155] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:22:32.662858 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a2f10-52ca-4aff-b127-349f75867a81 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:32.762076 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a2f10-52ca-4aff-b127-349f75867a81 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:32.794661 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a2d0c-55eb-495a-91f7-cf1fe3bd4a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:32.795461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a2d0c-55eb-495a-91f7-cf1fe3bd4a34 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:32.795768 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a2d0c-55eb-495a-91f7-cf1fe3bd4a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:32.796041 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a2d0c-55eb-495a-91f7-cf1fe3bd4a34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:32.803444 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:32.806536 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a2d0c-55eb-495a-91f7-cf1fe3bd4a34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:22:32.814924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-847a2d0c-55eb-495a-91f7-cf1fe3bd4a34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:22:32.814924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:32.814924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:32.814924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:32.814924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:32.814924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:32.814924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:32.814924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:32.814924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:32.814924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:32.814924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:32.814924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:32.814924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:22:32.814924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:22:32.814924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:22:32.814924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:22:32.816026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:22:32.816026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:22:32.816026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:22:32.816026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:22:32.816026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:32.816026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:32.816026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:32.816026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:32.816026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:32.816026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:32.816026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:32.816026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:32.816026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:32.816026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:32.816026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:32.816026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:32.817115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:32.817115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:32.817115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:32.817115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:32.817115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:22:32.817115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:22:32.817115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:22:32.817115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:22:32.817115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:22:32.817115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:22:32.817115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:22:32.817115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:22:32.817115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 570/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:22:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:22:32.822166 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd991661-d904-46a5-b486-9b5d00d76711 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:32.851472 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:32.919611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd991661-d904-46a5-b486-9b5d00d76711 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:32.959931 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e29d5-ed5d-4e06-932f-bee846f536e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:32.960602 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e29d5-ed5d-4e06-932f-bee846f536e4 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:32.960885 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e29d5-ed5d-4e06-932f-bee846f536e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:32.961157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e29d5-ed5d-4e06-932f-bee846f536e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:32.961900 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e29d5-ed5d-4e06-932f-bee846f536e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:32.962205 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e29d5-ed5d-4e06-932f-bee846f536e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:32.963684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e29d5-ed5d-4e06-932f-bee846f536e4 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:32.972331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 587/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:22:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:32.978109 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a2d0c-55eb-495a-91f7-cf1fe3bd4a34 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:33.075038 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a2d0c-55eb-495a-91f7-cf1fe3bd4a34 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:33.107936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7055d01-16d4-4270-aee9-329c7314554a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:33.108783 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7055d01-16d4-4270-aee9-329c7314554a None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:33.109062 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7055d01-16d4-4270-aee9-329c7314554a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:33.109332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7055d01-16d4-4270-aee9-329c7314554a None admin] PATH_INFO: `/v3/users/86d28122b6f4460eb4262976ead44b78` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:33.110493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7055d01-16d4-4270-aee9-329c7314554a None admin] RBAC: Authorizing `identity:get_user(user_id=86d28122b6f4460eb4262976ead44b78)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:33.111931 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7055d01-16d4-4270-aee9-329c7314554a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:33.113533 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 571/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:22:32 2022] GET /identity/v3/users/86d28122b6f4460eb4262976ead44b78 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:33.119346 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e29d5-ed5d-4e06-932f-bee846f536e4 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:33.210103 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:33.217459 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e29d5-ed5d-4e06-932f-bee846f536e4 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:33.249493 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40cf64-3efb-4875-ae35-43800ac24081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:33.250132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40cf64-3efb-4875-ae35-43800ac24081 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:33.250411 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40cf64-3efb-4875-ae35-43800ac24081 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:33.250684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40cf64-3efb-4875-ae35-43800ac24081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:33.320598 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40cf64-3efb-4875-ae35-43800ac24081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:33.322255 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40cf64-3efb-4875-ae35-43800ac24081 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:33.410759 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:33.418870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 588/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:22:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:22:33.424728 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7055d01-16d4-4270-aee9-329c7314554a None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:33.488687 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:33.520788 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7055d01-16d4-4270-aee9-329c7314554a None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:33.553517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1bea17-8b6b-455c-ad68-f08ee1a2aec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:33.554360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1bea17-8b6b-455c-ad68-f08ee1a2aec7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:33.554646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1bea17-8b6b-455c-ad68-f08ee1a2aec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:33.554923 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1bea17-8b6b-455c-ad68-f08ee1a2aec7 None admin] PATH_INFO: `/v3/projects/bdc190df4add446fa8da2d6b2d2edfeb` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:33.556113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1bea17-8b6b-455c-ad68-f08ee1a2aec7 None admin] RBAC: Authorizing `identity:get_project(project_id=bdc190df4add446fa8da2d6b2d2edfeb)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:33.557566 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1bea17-8b6b-455c-ad68-f08ee1a2aec7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:33.559280 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 572/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:22:33 2022] GET /identity/v3/projects/bdc190df4add446fa8da2d6b2d2edfeb => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:33.565312 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40cf64-3efb-4875-ae35-43800ac24081 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:33.662541 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40cf64-3efb-4875-ae35-43800ac24081 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:33.695078 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07658feb-9c5d-4f04-bdfb-bb9b681adb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:33.695727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07658feb-9c5d-4f04-bdfb-bb9b681adb5e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:33.696001 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07658feb-9c5d-4f04-bdfb-bb9b681adb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:33.696267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07658feb-9c5d-4f04-bdfb-bb9b681adb5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:33.697091 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07658feb-9c5d-4f04-bdfb-bb9b681adb5e None admin] RBAC: Adding query filter params (role.id=5ca112885467422684e134cb0f94865a, scope.project.id=bdc190df4add446fa8da2d6b2d2edfeb, user.id=86d28122b6f4460eb4262976ead44b78) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:33.697943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07658feb-9c5d-4f04-bdfb-bb9b681adb5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:33.699365 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07658feb-9c5d-4f04-bdfb-bb9b681adb5e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:33.705401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 589/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:22:33 2022] GET /identity/v3/role_assignments?user.id=86d28122b6f4460eb4262976ead44b78&scope.project.id=bdc190df4add446fa8da2d6b2d2edfeb&role.id=5ca112885467422684e134cb0f94865a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:35.712257 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968a7e1-7a08-487b-84e7-64bdd43ddf2d None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:35.712808 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968a7e1-7a08-487b-84e7-64bdd43ddf2d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:35.712808 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968a7e1-7a08-487b-84e7-64bdd43ddf2d None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:35.714209 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 573/1162] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:22:35.719541 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fd126-3ce2-4ca0-87c5-1adfa5c78a70 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:35.719934 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fd126-3ce2-4ca0-87c5-1adfa5c78a70 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:35.720106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fd126-3ce2-4ca0-87c5-1adfa5c78a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:35.737234 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-3b1fd126-3ce2-4ca0-87c5-1adfa5c78a70 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:35.738436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3b1fd126-3ce2-4ca0-87c5-1adfa5c78a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:35.791484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:35.810257 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 590/1163] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:22:35.831954 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968a7e1-7a08-487b-84e7-64bdd43ddf2d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:35.930650 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968a7e1-7a08-487b-84e7-64bdd43ddf2d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:35.963031 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15514718-9abd-4fc0-ba09-812f1d850059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:35.967090 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15514718-9abd-4fc0-ba09-812f1d850059 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:35.967090 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15514718-9abd-4fc0-ba09-812f1d850059 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:35.967090 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15514718-9abd-4fc0-ba09-812f1d850059 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:35.967822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:36.034523 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15514718-9abd-4fc0-ba09-812f1d850059 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:36.036029 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15514718-9abd-4fc0-ba09-812f1d850059 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:36.130738 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 574/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:22:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:22:36.136669 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fd126-3ce2-4ca0-87c5-1adfa5c78a70 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:36.233885 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fd126-3ce2-4ca0-87c5-1adfa5c78a70 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:36.267465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684cc0e2-8f47-4cd0-94f2-fdea16a4d367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:36.268267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684cc0e2-8f47-4cd0-94f2-fdea16a4d367 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:36.268548 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684cc0e2-8f47-4cd0-94f2-fdea16a4d367 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:36.269067 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684cc0e2-8f47-4cd0-94f2-fdea16a4d367 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:36.275950 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684cc0e2-8f47-4cd0-94f2-fdea16a4d367 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:36.277916 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684cc0e2-8f47-4cd0-94f2-fdea16a4d367 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:36.292343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-684cc0e2-8f47-4cd0-94f2-fdea16a4d367 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 20:22:36.292343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:36.292343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 14 20:22:36.292343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 14 20:22:36.292343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:22:36.292343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:22:36.292343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:22:36.292343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:22:36.292343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 14 20:22:36.292343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:36.292343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:22:36.292343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:36.292343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:36.292343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:36.292343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:36.292343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:36.292343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:36.293601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:36.293601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:36.293601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:36.293601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:36.293601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:36.293601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:36.293601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:22:36.293601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:22:36.293601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:22:36.293601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:22:36.293601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:36.293601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:36.293601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:36.293601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:36.293601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:36.293601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:36.294825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:36.294825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:36.294825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:36.294825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:36.294825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:36.294825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:36.294825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:36.294825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:36.294825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:36.294825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:36.294825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 14 20:22:36.294825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:22:36.294825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 20:22:36.294825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:36.294825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 591/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 14 20:22:36 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:22:36.299669 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15514718-9abd-4fc0-ba09-812f1d850059 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:36.332432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:36.396909 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15514718-9abd-4fc0-ba09-812f1d850059 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:36.429711 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8f110-f013-4414-bf44-658189fba677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:36.430356 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8f110-f013-4414-bf44-658189fba677 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:36.430636 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8f110-f013-4414-bf44-658189fba677 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:36.430909 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8f110-f013-4414-bf44-658189fba677 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:36.431650 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8f110-f013-4414-bf44-658189fba677 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:36.431952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8f110-f013-4414-bf44-658189fba677 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:36.433331 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8f110-f013-4414-bf44-658189fba677 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:36.440341 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 575/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 14 20:22:36 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:38.403678 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3c4b7-c427-47dc-a568-cf8a9968abc3 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:38.404200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3c4b7-c427-47dc-a568-cf8a9968abc3 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:38.404200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3c4b7-c427-47dc-a568-cf8a9968abc3 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:38.405571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 592/1167] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:22:38.410878 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca795bf6-e144-44e5-9006-8c7a8999bec6 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:38.411275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca795bf6-e144-44e5-9006-8c7a8999bec6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:38.411467 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca795bf6-e144-44e5-9006-8c7a8999bec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:38.427638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ca795bf6-e144-44e5-9006-8c7a8999bec6 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:38.428797 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ca795bf6-e144-44e5-9006-8c7a8999bec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:38.495568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 576/1168] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:22:38.518025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3c4b7-c427-47dc-a568-cf8a9968abc3 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:38.566517 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:38.616736 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3c4b7-c427-47dc-a568-cf8a9968abc3 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:38.650968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28b154-c3b0-450e-9f33-d0d4ce170d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:38.651640 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28b154-c3b0-450e-9f33-d0d4ce170d6c None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:38.651922 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28b154-c3b0-450e-9f33-d0d4ce170d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:38.652197 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28b154-c3b0-450e-9f33-d0d4ce170d6c None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:38.652948 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28b154-c3b0-450e-9f33-d0d4ce170d6c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:38.654357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28b154-c3b0-450e-9f33-d0d4ce170d6c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:38.668528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 593/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Thu Jul 14 20:22:38 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:22:40.701945 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2509025-05a1-42fe-badf-0bb7114bcd76 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:40.702520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2509025-05a1-42fe-badf-0bb7114bcd76 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:40.702520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2509025-05a1-42fe-badf-0bb7114bcd76 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:40.703905 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 577/1170] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:22:40.709670 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b15e5-0c14-466a-8c56-b4a1874db2ba None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:40.710114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b15e5-0c14-466a-8c56-b4a1874db2ba None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:40.710255 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b15e5-0c14-466a-8c56-b4a1874db2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:40.727124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-264b15e5-0c14-466a-8c56-b4a1874db2ba None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:40.728398 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-264b15e5-0c14-466a-8c56-b4a1874db2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:40.793671 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 594/1171] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:22:40.816584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2509025-05a1-42fe-badf-0bb7114bcd76 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:40.846617 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:40.917741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2509025-05a1-42fe-badf-0bb7114bcd76 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:40.951175 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565e326-f52a-44e6-adc3-29f092bce818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:40.952031 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565e326-f52a-44e6-adc3-29f092bce818 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:40.952318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565e326-f52a-44e6-adc3-29f092bce818 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:40.952591 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565e326-f52a-44e6-adc3-29f092bce818 None admin] PATH_INFO: `/v3/domains/d226580e8d734281939c2c345698f4b5` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:40.962647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565e326-f52a-44e6-adc3-29f092bce818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d226580e8d734281939c2c345698f4b5)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:40.964720 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565e326-f52a-44e6-adc3-29f092bce818 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:40.966347 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 578/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Thu Jul 14 20:22:40 2022] GET /identity/v3/domains/d226580e8d734281939c2c345698f4b5 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:40.972757 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b15e5-0c14-466a-8c56-b4a1874db2ba None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:41.009903 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:41.069969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b15e5-0c14-466a-8c56-b4a1874db2ba None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:41.103701 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76beb1-5c69-4543-afdc-986c3c34fef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:41.104387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76beb1-5c69-4543-afdc-986c3c34fef5 None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:41.104671 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76beb1-5c69-4543-afdc-986c3c34fef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:41.104943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76beb1-5c69-4543-afdc-986c3c34fef5 None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:41.105646 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76beb1-5c69-4543-afdc-986c3c34fef5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:41.107105 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76beb1-5c69-4543-afdc-986c3c34fef5 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:41.130756 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 595/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 14 20:22:40 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:22:43.144311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81acfa-72a4-475f-874d-410c46979d75 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:43.144862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81acfa-72a4-475f-874d-410c46979d75 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:43.144862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81acfa-72a4-475f-874d-410c46979d75 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:43.146206 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 579/1174] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:22:43.151271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357c4d4-0879-4002-aeaf-e85cbc2533b3 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:43.151713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357c4d4-0879-4002-aeaf-e85cbc2533b3 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:43.151944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357c4d4-0879-4002-aeaf-e85cbc2533b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:43.168851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-6357c4d4-0879-4002-aeaf-e85cbc2533b3 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:43.170042 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6357c4d4-0879-4002-aeaf-e85cbc2533b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:43.235523 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 596/1175] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:22:43.259883 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81acfa-72a4-475f-874d-410c46979d75 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:43.348710 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:43.359896 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81acfa-72a4-475f-874d-410c46979d75 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:43.392524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e470aa1-fb96-42af-a70e-78d726b25f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:43.393371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e470aa1-fb96-42af-a70e-78d726b25f7b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:43.393654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e470aa1-fb96-42af-a70e-78d726b25f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:43.394183 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e470aa1-fb96-42af-a70e-78d726b25f7b None admin] PATH_INFO: `/v3/domains/d226580e8d734281939c2c345698f4b5` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:43.395428 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e470aa1-fb96-42af-a70e-78d726b25f7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=d226580e8d734281939c2c345698f4b5)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:43.396802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e470aa1-fb96-42af-a70e-78d726b25f7b None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:43.398407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 580/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Thu Jul 14 20:22:43 2022] GET /identity/v3/domains/d226580e8d734281939c2c345698f4b5 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:43.404307 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357c4d4-0879-4002-aeaf-e85cbc2533b3 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:43.425613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:43.502238 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357c4d4-0879-4002-aeaf-e85cbc2533b3 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:43.534425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a04d1-fd08-4e43-ac12-740b17bc8e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:43.535107 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a04d1-fd08-4e43-ac12-740b17bc8e46 None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:43.535393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a04d1-fd08-4e43-ac12-740b17bc8e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:43.535670 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a04d1-fd08-4e43-ac12-740b17bc8e46 None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:43.536370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a04d1-fd08-4e43-ac12-740b17bc8e46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:43.537793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a04d1-fd08-4e43-ac12-740b17bc8e46 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:43.558256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 597/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 14 20:22:43 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:22:45.599018 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2545b-cf99-43db-894a-3347a7fbd6ed None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:45.599570 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2545b-cf99-43db-894a-3347a7fbd6ed None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:45.599570 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2545b-cf99-43db-894a-3347a7fbd6ed None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:45.600923 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 581/1178] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:22:45.607029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383c47d-e784-46d3-aba7-90b5bdcc0965 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:45.607458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383c47d-e784-46d3-aba7-90b5bdcc0965 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:45.607583 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383c47d-e784-46d3-aba7-90b5bdcc0965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:45.624593 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-0383c47d-e784-46d3-aba7-90b5bdcc0965 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:45.625767 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-0383c47d-e784-46d3-aba7-90b5bdcc0965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:45.692102 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 598/1179] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:22:45.715212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2545b-cf99-43db-894a-3347a7fbd6ed None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:45.813538 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2545b-cf99-43db-894a-3347a7fbd6ed None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:45.845879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db338957-eaf7-454b-bd7f-9b25ce191f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:45.846651 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db338957-eaf7-454b-bd7f-9b25ce191f58 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:45.846930 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db338957-eaf7-454b-bd7f-9b25ce191f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:45.847230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db338957-eaf7-454b-bd7f-9b25ce191f58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:45.857747 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db338957-eaf7-454b-bd7f-9b25ce191f58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:22:45.865165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-db338957-eaf7-454b-bd7f-9b25ce191f58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:22:45.865165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:45.865165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:45.865165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:45.865165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:45.865165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:45.865165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:45.865165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:45.865165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:45.865165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:45.865165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:45.865165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:45.865165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:22:45.865165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:22:45.865165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:22:45.865165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:22:45.866303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:22:45.866303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:22:45.866303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:22:45.866303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:22:45.866303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:45.866303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:45.866303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:45.866303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:45.866303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:45.866303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:45.866303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:45.866303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:45.866303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:45.866303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:45.866303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:45.866303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:45.867577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:45.867577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:45.867577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:45.867577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:45.867577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:22:45.867577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:22:45.867577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:22:45.867577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:22:45.867577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:22:45.867577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:22:45.867577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:22:45.867577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:22:45.867577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 582/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:22:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:22:45.872443 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383c47d-e784-46d3-aba7-90b5bdcc0965 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:45.880600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:45.969173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383c47d-e784-46d3-aba7-90b5bdcc0965 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:46.001763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fc763-5d6c-43dd-a6a1-a9073454a76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:46.002425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fc763-5d6c-43dd-a6a1-a9073454a76a None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:46.002710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fc763-5d6c-43dd-a6a1-a9073454a76a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:46.003014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fc763-5d6c-43dd-a6a1-a9073454a76a None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:46.003762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fc763-5d6c-43dd-a6a1-a9073454a76a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:46.004069 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fc763-5d6c-43dd-a6a1-a9073454a76a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:46.005418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fc763-5d6c-43dd-a6a1-a9073454a76a None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:46.013253 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 599/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:22:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:46.019398 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db338957-eaf7-454b-bd7f-9b25ce191f58 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:46.038835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:46.116935 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db338957-eaf7-454b-bd7f-9b25ce191f58 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:46.148775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270a7fd-4f61-4b38-aa05-0df9e7845326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:46.149632 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270a7fd-4f61-4b38-aa05-0df9e7845326 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:46.149916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270a7fd-4f61-4b38-aa05-0df9e7845326 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:46.150190 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270a7fd-4f61-4b38-aa05-0df9e7845326 None admin] PATH_INFO: `/v3/users/8c48a2424d224b48b1128b5553bb2c9c` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:46.163367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270a7fd-4f61-4b38-aa05-0df9e7845326 None admin] RBAC: Authorizing `identity:get_user(user_id=8c48a2424d224b48b1128b5553bb2c9c)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:46.164799 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270a7fd-4f61-4b38-aa05-0df9e7845326 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:46.166970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 583/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:22:46 2022] GET /identity/v3/users/8c48a2424d224b48b1128b5553bb2c9c => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:46.172762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fc763-5d6c-43dd-a6a1-a9073454a76a None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:46.237197 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:46.270057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fc763-5d6c-43dd-a6a1-a9073454a76a None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:46.302898 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc61e04-9b21-4a5a-b995-39686343b4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:46.303577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc61e04-9b21-4a5a-b995-39686343b4de None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:46.303899 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc61e04-9b21-4a5a-b995-39686343b4de None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:46.304180 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc61e04-9b21-4a5a-b995-39686343b4de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:46.376026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc61e04-9b21-4a5a-b995-39686343b4de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:46.377382 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc61e04-9b21-4a5a-b995-39686343b4de None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:46.439563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:46.473244 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 600/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:22:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:22:46.479184 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270a7fd-4f61-4b38-aa05-0df9e7845326 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:46.554569 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:46.578161 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270a7fd-4f61-4b38-aa05-0df9e7845326 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:46.611065 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a458d5d-4c78-452b-aae7-9ecf282392ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:46.611907 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a458d5d-4c78-452b-aae7-9ecf282392ab None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:46.612183 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a458d5d-4c78-452b-aae7-9ecf282392ab None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:46.612454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a458d5d-4c78-452b-aae7-9ecf282392ab None admin] PATH_INFO: `/v3/projects/e1f82f5d367e43549ecdd75b054e2486` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:46.613600 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a458d5d-4c78-452b-aae7-9ecf282392ab None admin] RBAC: Authorizing `identity:get_project(project_id=e1f82f5d367e43549ecdd75b054e2486)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:46.615046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a458d5d-4c78-452b-aae7-9ecf282392ab None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:46.616717 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 584/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:22:46 2022] GET /identity/v3/projects/e1f82f5d367e43549ecdd75b054e2486 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:46.623415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc61e04-9b21-4a5a-b995-39686343b4de None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:46.719795 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc61e04-9b21-4a5a-b995-39686343b4de None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:46.752595 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d1ce2-a3bf-4101-938e-94f82ac93ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:46.753823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d1ce2-a3bf-4101-938e-94f82ac93ecb None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:46.753823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d1ce2-a3bf-4101-938e-94f82ac93ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:46.753823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d1ce2-a3bf-4101-938e-94f82ac93ecb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:46.754574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d1ce2-a3bf-4101-938e-94f82ac93ecb None admin] RBAC: Adding query filter params (role.id=5ca112885467422684e134cb0f94865a, scope.project.id=e1f82f5d367e43549ecdd75b054e2486, user.id=8c48a2424d224b48b1128b5553bb2c9c) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:46.754887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d1ce2-a3bf-4101-938e-94f82ac93ecb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:46.756337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d1ce2-a3bf-4101-938e-94f82ac93ecb None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:46.762261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 601/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:22:46 2022] GET /identity/v3/role_assignments?user.id=8c48a2424d224b48b1128b5553bb2c9c&scope.project.id=e1f82f5d367e43549ecdd75b054e2486&role.id=5ca112885467422684e134cb0f94865a => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:48.745860 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75893b-000b-4c89-b221-94394646346a None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:48.746410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75893b-000b-4c89-b221-94394646346a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:48.746410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75893b-000b-4c89-b221-94394646346a None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:48.747811 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 585/1186] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:22:48.753122 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5310757-c04b-4ec2-9bf5-386c63ab853a None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:48.753539 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5310757-c04b-4ec2-9bf5-386c63ab853a None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:48.753691 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5310757-c04b-4ec2-9bf5-386c63ab853a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:48.770928 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-e5310757-c04b-4ec2-9bf5-386c63ab853a None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:48.772129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e5310757-c04b-4ec2-9bf5-386c63ab853a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:48.797781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:48.838536 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 602/1187] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:22:48.860336 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75893b-000b-4c89-b221-94394646346a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:48.959148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75893b-000b-4c89-b221-94394646346a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:48.990797 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a6798-e32b-4c07-8b1d-912adf646e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:48.991597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a6798-e32b-4c07-8b1d-912adf646e3b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:48.991878 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a6798-e32b-4c07-8b1d-912adf646e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:48.992147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a6798-e32b-4c07-8b1d-912adf646e3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:49.002689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a6798-e32b-4c07-8b1d-912adf646e3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:22:49.010983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-fb6a6798-e32b-4c07-8b1d-912adf646e3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:22:49.010983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:49.010983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:49.010983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:49.010983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:49.010983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:49.010983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:49.010983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:49.010983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:49.010983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:49.010983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:49.010983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:49.010983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:22:49.010983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:22:49.010983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:22:49.010983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:22:49.012045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:22:49.012045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:22:49.012045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:22:49.012045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:22:49.012045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:49.012045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:49.012045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:49.012045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:49.012045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:49.012045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:49.012045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:49.012045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:49.012045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:49.012045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:49.012045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:49.012045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:49.012938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:49.012938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:49.012938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:49.012938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:49.012938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:22:49.012938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:22:49.012938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:22:49.012938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:22:49.012938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:22:49.012938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:22:49.012938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:22:49.012938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:22:49.012938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 586/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:22:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:22:49.018338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5310757-c04b-4ec2-9bf5-386c63ab853a None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:49.112742 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5310757-c04b-4ec2-9bf5-386c63ab853a None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:49.146367 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9459ea-bc06-4682-be41-01aba543a383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:49.147050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9459ea-bc06-4682-be41-01aba543a383 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:49.147331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9459ea-bc06-4682-be41-01aba543a383 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:49.147603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9459ea-bc06-4682-be41-01aba543a383 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:49.148341 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9459ea-bc06-4682-be41-01aba543a383 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:49.148642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9459ea-bc06-4682-be41-01aba543a383 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:49.149989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9459ea-bc06-4682-be41-01aba543a383 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:49.157765 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 603/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:22:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:49.163562 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a6798-e32b-4c07-8b1d-912adf646e3b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:49.167826 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:49.260925 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a6798-e32b-4c07-8b1d-912adf646e3b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:49.293315 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db12c57-2217-404d-a917-c5501dd3a6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:49.294181 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db12c57-2217-404d-a917-c5501dd3a6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:49.294466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db12c57-2217-404d-a917-c5501dd3a6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:49.294742 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db12c57-2217-404d-a917-c5501dd3a6b4 None admin] PATH_INFO: `/v3/users/8c48a2424d224b48b1128b5553bb2c9c` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:49.296025 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db12c57-2217-404d-a917-c5501dd3a6b4 None admin] RBAC: Authorizing `identity:get_user(user_id=8c48a2424d224b48b1128b5553bb2c9c)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:49.297554 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db12c57-2217-404d-a917-c5501dd3a6b4 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:49.299261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 587/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:22:49 2022] GET /identity/v3/users/8c48a2424d224b48b1128b5553bb2c9c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:49.305053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9459ea-bc06-4682-be41-01aba543a383 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:49.330177 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:49.403233 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9459ea-bc06-4682-be41-01aba543a383 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:49.435959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f70090-9a9b-4dbc-85bf-d9815a218cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:49.436812 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f70090-9a9b-4dbc-85bf-d9815a218cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:49.437093 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f70090-9a9b-4dbc-85bf-d9815a218cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:49.437366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f70090-9a9b-4dbc-85bf-d9815a218cc7 None admin] PATH_INFO: `/v3/projects/e1f82f5d367e43549ecdd75b054e2486` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:49.438532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f70090-9a9b-4dbc-85bf-d9815a218cc7 None admin] RBAC: Authorizing `identity:get_project(project_id=e1f82f5d367e43549ecdd75b054e2486)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:49.439993 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f70090-9a9b-4dbc-85bf-d9815a218cc7 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:49.441688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 604/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:22:49 2022] GET /identity/v3/projects/e1f82f5d367e43549ecdd75b054e2486 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:49.447816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db12c57-2217-404d-a917-c5501dd3a6b4 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:49.515736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:49.545076 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db12c57-2217-404d-a917-c5501dd3a6b4 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:49.578767 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be660a7-d45a-4aeb-ac8b-1719400e3b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:49.579687 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be660a7-d45a-4aeb-ac8b-1719400e3b96 None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:49.579970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be660a7-d45a-4aeb-ac8b-1719400e3b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:49.580245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be660a7-d45a-4aeb-ac8b-1719400e3b96 None admin] PATH_INFO: `/v3/projects/e1f82f5d367e43549ecdd75b054e2486/users/8c48a2424d224b48b1128b5553bb2c9c/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:49.582178 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be660a7-d45a-4aeb-ac8b-1719400e3b96 None admin] RBAC: Authorizing `identity:create_grant(project_id=e1f82f5d367e43549ecdd75b054e2486, user_id=8c48a2424d224b48b1128b5553bb2c9c, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:49.583640 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be660a7-d45a-4aeb-ac8b-1719400e3b96 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:49.596471 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 588/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:22:49 2022] PUT /identity/v3/projects/e1f82f5d367e43549ecdd75b054e2486/users/8c48a2424d224b48b1128b5553bb2c9c/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:22:51.577968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed29d6e-d6b2-4c48-ae4d-7cb3854333de None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:51.578492 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed29d6e-d6b2-4c48-ae4d-7cb3854333de None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:51.578599 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed29d6e-d6b2-4c48-ae4d-7cb3854333de None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:51.579941 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 605/1193] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:22:51.585198 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603351db-deb8-42c3-aee3-fab2ab7131ba None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:51.585587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603351db-deb8-42c3-aee3-fab2ab7131ba None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:51.585761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603351db-deb8-42c3-aee3-fab2ab7131ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:51.602680 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-603351db-deb8-42c3-aee3-fab2ab7131ba None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:22:51.603927 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-603351db-deb8-42c3-aee3-fab2ab7131ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:22:51.675458 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 589/1194] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:22:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:22:51.698767 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed29d6e-d6b2-4c48-ae4d-7cb3854333de None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:51.794507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:51.804566 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed29d6e-d6b2-4c48-ae4d-7cb3854333de None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:51.837779 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d69156-ad6f-4977-9d22-fc8d937f5f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:51.838508 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d69156-ad6f-4977-9d22-fc8d937f5f90 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:51.838793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d69156-ad6f-4977-9d22-fc8d937f5f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:51.839103 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d69156-ad6f-4977-9d22-fc8d937f5f90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:51.849634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d69156-ad6f-4977-9d22-fc8d937f5f90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:22:51.857293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-92d69156-ad6f-4977-9d22-fc8d937f5f90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:22:51.857293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:22:51.857293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:22:51.857293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:22:51.857293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:22:51.857293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:22:51.857293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:22:51.857293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:22:51.857293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:22:51.857293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:22:51.857293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:22:51.857293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:22:51.857293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:22:51.857293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:22:51.857293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:22:51.857293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:22:51.858413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:22:51.858413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:22:51.858413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:22:51.858413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:22:51.858413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:22:51.858413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:22:51.858413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:22:51.858413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:22:51.858413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:22:51.858413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:22:51.858413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:22:51.858413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:22:51.858413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:22:51.858413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:22:51.858413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:22:51.858413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:22:51.859604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:22:51.859604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:22:51.859604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:22:51.859604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:22:51.859604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:22:51.859604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:22:51.859604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:22:51.859604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:22:51.859604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:22:51.859604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:22:51.859604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:22:51.859604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:22:51.859604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 606/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:22:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:22:51.864764 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603351db-deb8-42c3-aee3-fab2ab7131ba None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:51.925134 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:51.963908 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603351db-deb8-42c3-aee3-fab2ab7131ba None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:51.996743 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86325b00-4990-4868-9331-17b323a64b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:51.997402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86325b00-4990-4868-9331-17b323a64b96 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:51.997686 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86325b00-4990-4868-9331-17b323a64b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:51.997962 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86325b00-4990-4868-9331-17b323a64b96 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:51.998704 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86325b00-4990-4868-9331-17b323a64b96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:51.999042 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86325b00-4990-4868-9331-17b323a64b96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:52.000413 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86325b00-4990-4868-9331-17b323a64b96 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:52.008306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 590/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:22:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:52.014778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d69156-ad6f-4977-9d22-fc8d937f5f90 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:52.109737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d69156-ad6f-4977-9d22-fc8d937f5f90 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:52.142784 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b27dc7-ae99-43a7-9c6c-59a070ceb03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:52.143672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b27dc7-ae99-43a7-9c6c-59a070ceb03d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:52.144207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b27dc7-ae99-43a7-9c6c-59a070ceb03d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:52.144481 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b27dc7-ae99-43a7-9c6c-59a070ceb03d None admin] PATH_INFO: `/v3/users/8c48a2424d224b48b1128b5553bb2c9c` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:52.145649 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b27dc7-ae99-43a7-9c6c-59a070ceb03d None admin] RBAC: Authorizing `identity:get_user(user_id=8c48a2424d224b48b1128b5553bb2c9c)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:52.147109 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b27dc7-ae99-43a7-9c6c-59a070ceb03d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:52.148721 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 607/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 14 20:22:52 2022] GET /identity/v3/users/8c48a2424d224b48b1128b5553bb2c9c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:52.154436 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86325b00-4990-4868-9331-17b323a64b96 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:52.251109 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86325b00-4990-4868-9331-17b323a64b96 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:52.272848 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:52.283463 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e07ddb-5b4f-4b99-905d-3af2691d873d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:52.284113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e07ddb-5b4f-4b99-905d-3af2691d873d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:52.284396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e07ddb-5b4f-4b99-905d-3af2691d873d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:52.284671 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e07ddb-5b4f-4b99-905d-3af2691d873d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:52.349799 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e07ddb-5b4f-4b99-905d-3af2691d873d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:52.351271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e07ddb-5b4f-4b99-905d-3af2691d873d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:52.445980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 591/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:22:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:22:52.451877 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b27dc7-ae99-43a7-9c6c-59a070ceb03d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:52.472849 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:52.548854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b27dc7-ae99-43a7-9c6c-59a070ceb03d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:52.580544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229b94a-c891-44ee-9282-21cc1b685f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:52.581741 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229b94a-c891-44ee-9282-21cc1b685f98 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:52.581741 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229b94a-c891-44ee-9282-21cc1b685f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:52.582081 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229b94a-c891-44ee-9282-21cc1b685f98 None admin] PATH_INFO: `/v3/projects/e1f82f5d367e43549ecdd75b054e2486` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:52.583117 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1229b94a-c891-44ee-9282-21cc1b685f98 None admin] RBAC: Authorizing `identity:get_project(project_id=e1f82f5d367e43549ecdd75b054e2486)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:52.584534 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1229b94a-c891-44ee-9282-21cc1b685f98 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:52.586225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 608/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 14 20:22:52 2022] GET /identity/v3/projects/e1f82f5d367e43549ecdd75b054e2486 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:22:52.593325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e07ddb-5b4f-4b99-905d-3af2691d873d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:22:52.617775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:22:52.690724 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e07ddb-5b4f-4b99-905d-3af2691d873d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:22:52.723969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca1406-1aca-4903-ac3d-8e42f9d86e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:22:52.724538 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca1406-1aca-4903-ac3d-8e42f9d86e47 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:22:52.724823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca1406-1aca-4903-ac3d-8e42f9d86e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:22:52.725102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca1406-1aca-4903-ac3d-8e42f9d86e47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:22:52.725920 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca1406-1aca-4903-ac3d-8e42f9d86e47 None admin] RBAC: Adding query filter params (role.id=5ca112885467422684e134cb0f94865a, scope.project.id=e1f82f5d367e43549ecdd75b054e2486, user.id=8c48a2424d224b48b1128b5553bb2c9c) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:22:52.726230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca1406-1aca-4903-ac3d-8e42f9d86e47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:22:52.727704 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca1406-1aca-4903-ac3d-8e42f9d86e47 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:22:52.733930 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 592/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:22:52 2022] GET /identity/v3/role_assignments?user.id=8c48a2424d224b48b1128b5553bb2c9c&scope.project.id=e1f82f5d367e43549ecdd75b054e2486&role.id=5ca112885467422684e134cb0f94865a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:23:02.576547 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=56 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:02.731340 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=56 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:48.799261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab2a29-5157-422e-ac34-318522d5e06d None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:48.799840 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab2a29-5157-422e-ac34-318522d5e06d None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:48.799840 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab2a29-5157-422e-ac34-318522d5e06d None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:48.801193 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 609/1201] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:23:48.807444 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818ec33-8b60-4939-9006-82ac8e4dcd32 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:48.807997 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818ec33-8b60-4939-9006-82ac8e4dcd32 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:48.807997 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818ec33-8b60-4939-9006-82ac8e4dcd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:48.828100 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e818ec33-8b60-4939-9006-82ac8e4dcd32 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:23:48.829321 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e818ec33-8b60-4939-9006-82ac8e4dcd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:23:48.912618 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 593/1202] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:23:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:23:48.936517 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab2a29-5157-422e-ac34-318522d5e06d None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:49.089283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab2a29-5157-422e-ac34-318522d5e06d None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:49.135384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1193cf-742b-49b3-9c06-95c2bda74adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:49.136307 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1193cf-742b-49b3-9c06-95c2bda74adc None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:49.136655 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1193cf-742b-49b3-9c06-95c2bda74adc None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:49.137011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1193cf-742b-49b3-9c06-95c2bda74adc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:49.152567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1193cf-742b-49b3-9c06-95c2bda74adc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:49.154383 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1193cf-742b-49b3-9c06-95c2bda74adc None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:49.183247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-ec1193cf-742b-49b3-9c06-95c2bda74adc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:23:49.183247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:23:49.183247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:23:49.183247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:23:49.183247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:23:49.183247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:23:49.183247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:23:49.183247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:23:49.183247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:23:49.183247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:23:49.183247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:23:49.183247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:23:49.183247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:23:49.183247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:23:49.183247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:23:49.185304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:23:49.185304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:23:49.185304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:23:49.185304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:23:49.185304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:23:49.185304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:23:49.185304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:23:49.185304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:23:49.185304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:23:49.185304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:23:49.185304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:23:49.185304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:23:49.185304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:23:49.185304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:23:49.185304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:23:49.185304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:23:49.185304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:23:49.193800 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:23:49.193800 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:23:49.193800 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:23:49.193800 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:23:49.193800 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 610/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:23:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:23:49.198596 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818ec33-8b60-4939-9006-82ac8e4dcd32 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:49.364918 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818ec33-8b60-4939-9006-82ac8e4dcd32 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:49.408126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:49.417173 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43d8b4-e05d-4167-9b7b-233bc0141a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:49.417871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43d8b4-e05d-4167-9b7b-233bc0141a80 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:49.418152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43d8b4-e05d-4167-9b7b-233bc0141a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:49.418424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43d8b4-e05d-4167-9b7b-233bc0141a80 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:49.419267 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43d8b4-e05d-4167-9b7b-233bc0141a80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:23:49.419554 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43d8b4-e05d-4167-9b7b-233bc0141a80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:49.421246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43d8b4-e05d-4167-9b7b-233bc0141a80 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:49.439919 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 594/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:23:49.448621 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1193cf-742b-49b3-9c06-95c2bda74adc None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:49.515721 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:49.580532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1193cf-742b-49b3-9c06-95c2bda74adc None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:49.632053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf8cf2-90d5-4034-ac32-d6a42c96fad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:49.632972 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf8cf2-90d5-4034-ac32-d6a42c96fad8 None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:49.633297 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf8cf2-90d5-4034-ac32-d6a42c96fad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:49.633574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf8cf2-90d5-4034-ac32-d6a42c96fad8 None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:49.636800 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf8cf2-90d5-4034-ac32-d6a42c96fad8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:49.636800 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf8cf2-90d5-4034-ac32-d6a42c96fad8 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:49.664845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 611/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 14 20:23:49 2022] POST /identity/v3/users => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:23:52.202005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498ffbf-fff0-4e6a-8967-8d82d2421bfa None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:52.202556 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498ffbf-fff0-4e6a-8967-8d82d2421bfa None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:52.202968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498ffbf-fff0-4e6a-8967-8d82d2421bfa None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:52.204205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 595/1206] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:23:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:23:52.213368 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c16f59-262d-4cd5-a450-9efd19aa49a1 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:52.213368 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c16f59-262d-4cd5-a450-9efd19aa49a1 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:52.213368 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c16f59-262d-4cd5-a450-9efd19aa49a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:52.235722 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-e1c16f59-262d-4cd5-a450-9efd19aa49a1 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:23:52.237661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e1c16f59-262d-4cd5-a450-9efd19aa49a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:23:52.325111 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 612/1207] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:23:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:23:52.351183 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498ffbf-fff0-4e6a-8967-8d82d2421bfa None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:52.468872 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498ffbf-fff0-4e6a-8967-8d82d2421bfa None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:52.510728 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d85fb-d8d2-4bcd-83fe-ee6ac4dabb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:52.511613 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d85fb-d8d2-4bcd-83fe-ee6ac4dabb9b None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:52.511987 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d85fb-d8d2-4bcd-83fe-ee6ac4dabb9b None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:52.512179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d85fb-d8d2-4bcd-83fe-ee6ac4dabb9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:52.523476 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d85fb-d8d2-4bcd-83fe-ee6ac4dabb9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:23:52.532012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-562d85fb-d8d2-4bcd-83fe-ee6ac4dabb9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:23:52.532012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:23:52.532012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:23:52.532012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:23:52.532012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:23:52.532012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:23:52.532012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:23:52.532012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:23:52.532012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:23:52.532012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:23:52.532012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:23:52.532012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:23:52.532012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:23:52.532012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:23:52.532012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:23:52.532012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:23:52.533103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:23:52.533103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:23:52.533103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:23:52.533103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:23:52.533103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:23:52.533103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:23:52.533103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:23:52.533103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:23:52.533103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:23:52.533103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:23:52.533103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:23:52.533103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:23:52.533103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:23:52.533103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:23:52.533103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:23:52.533103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:23:52.534149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:23:52.534149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:23:52.534149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:23:52.534149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:23:52.534149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:23:52.534149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:23:52.534149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:23:52.534149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:23:52.534149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:23:52.534149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:23:52.534149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:23:52.534149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:23:52.534149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 596/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:23:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:23:52.539528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c16f59-262d-4cd5-a450-9efd19aa49a1 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:52.592019 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:52.667099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c16f59-262d-4cd5-a450-9efd19aa49a1 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:52.706558 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba445d25-92b6-4e66-838c-d903aed44db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:52.707275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba445d25-92b6-4e66-838c-d903aed44db9 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:52.707565 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba445d25-92b6-4e66-838c-d903aed44db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:52.707883 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba445d25-92b6-4e66-838c-d903aed44db9 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:52.708626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba445d25-92b6-4e66-838c-d903aed44db9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:23:52.708919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba445d25-92b6-4e66-838c-d903aed44db9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:52.710299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba445d25-92b6-4e66-838c-d903aed44db9 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:52.723984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 613/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:23:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:23:52.728078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d85fb-d8d2-4bcd-83fe-ee6ac4dabb9b None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:52.784650 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:52.843230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d85fb-d8d2-4bcd-83fe-ee6ac4dabb9b None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:52.884937 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0bee0-7f07-427a-b085-ba0a3ca07160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:52.885720 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0bee0-7f07-427a-b085-ba0a3ca07160 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:52.886008 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0bee0-7f07-427a-b085-ba0a3ca07160 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:52.886286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0bee0-7f07-427a-b085-ba0a3ca07160 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:52.898383 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0bee0-7f07-427a-b085-ba0a3ca07160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:52.899991 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0bee0-7f07-427a-b085-ba0a3ca07160 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:52.918670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-34a0bee0-7f07-427a-b085-ba0a3ca07160 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:23:52.918670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:23:52.918670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:23:52.918670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:23:52.918670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:23:52.918670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:23:52.918670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:23:52.918670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:23:52.918670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:23:52.918670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:23:52.918670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:23:52.918670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:23:52.918670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:23:52.918670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:23:52.918670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:23:52.919956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:23:52.919956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:23:52.919956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:23:52.919956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:23:52.919956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:23:52.919956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:23:52.919956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:23:52.919956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:23:52.919956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:23:52.919956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:23:52.919956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:23:52.919956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:23:52.919956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:23:52.919956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:23:52.919956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:23:52.919956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:23:52.919956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:23:52.921144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:23:52.921144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:23:52.921144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:23:52.921144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:23:52.921144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 597/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:23:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:23:52.925987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba445d25-92b6-4e66-838c-d903aed44db9 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:53.041863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:53.045001 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba445d25-92b6-4e66-838c-d903aed44db9 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:53.083581 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac734d-d216-4927-8391-50a7dd36f873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:53.084246 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac734d-d216-4927-8391-50a7dd36f873 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:53.084529 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac734d-d216-4927-8391-50a7dd36f873 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:53.084804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac734d-d216-4927-8391-50a7dd36f873 None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:53.085537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac734d-d216-4927-8391-50a7dd36f873 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:23:53.085848 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac734d-d216-4927-8391-50a7dd36f873 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:53.087277 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac734d-d216-4927-8391-50a7dd36f873 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:53.100740 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 614/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:23:53.107644 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0bee0-7f07-427a-b085-ba0a3ca07160 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:53.221269 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0bee0-7f07-427a-b085-ba0a3ca07160 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:53.223975 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:53.259328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b5467-9503-480b-a87a-9b00592a9a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:53.260136 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b5467-9503-480b-a87a-9b00592a9a04 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:53.260422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b5467-9503-480b-a87a-9b00592a9a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:53.260715 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b5467-9503-480b-a87a-9b00592a9a04 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:53.269950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b5467-9503-480b-a87a-9b00592a9a04 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:53.271438 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b5467-9503-480b-a87a-9b00592a9a04 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:53.288394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-7b6b5467-9503-480b-a87a-9b00592a9a04 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 20:23:53.288394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:23:53.288394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:23:53.288394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:23:53.288394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:23:53.288394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:23:53.288394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:23:53.288394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:23:53.288394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:23:53.288394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:23:53.288394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:23:53.288394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:23:53.288394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:23:53.288394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:23:53.288394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:23:53.288394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:23:53.289450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:23:53.289450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:23:53.289450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:23:53.289450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:23:53.289450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:23:53.289450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:23:53.289450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:23:53.289450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:23:53.289450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:23:53.289450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:23:53.289450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:23:53.289450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:23:53.289450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:23:53.289450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:23:53.289450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:23:53.289450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:23:53.289450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:23:53.289450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:23:53.291261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:23:53.291261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:23:53.291261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:23:53.291261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:23:53.291261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:23:53.291261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:23:53.291261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:23:53.291261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:23:53.291261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 20:23:53.291261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:23:53.295143 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 598/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:23:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:23:53.298439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac734d-d216-4927-8391-50a7dd36f873 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:53.417713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac734d-d216-4927-8391-50a7dd36f873 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:53.453502 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6e7b5-feeb-4b2c-bef4-013a718ec40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:53.454205 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6e7b5-feeb-4b2c-bef4-013a718ec40c None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:53.454496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6e7b5-feeb-4b2c-bef4-013a718ec40c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:53.454773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6e7b5-feeb-4b2c-bef4-013a718ec40c None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:53.456154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6e7b5-feeb-4b2c-bef4-013a718ec40c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:23:53.456467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6e7b5-feeb-4b2c-bef4-013a718ec40c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:53.457896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6e7b5-feeb-4b2c-bef4-013a718ec40c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:53.482105 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 615/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jul 14 20:23:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:23:53.483293 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b5467-9503-480b-a87a-9b00592a9a04 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:53.604014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b5467-9503-480b-a87a-9b00592a9a04 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:53.643596 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1541c-15b0-4310-a898-b248b5c2058e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:53.644322 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1541c-15b0-4310-a898-b248b5c2058e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:53.644702 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1541c-15b0-4310-a898-b248b5c2058e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:53.645102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1541c-15b0-4310-a898-b248b5c2058e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:53.679864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:53.724333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1541c-15b0-4310-a898-b248b5c2058e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:53.725812 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1541c-15b0-4310-a898-b248b5c2058e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:53.839097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 599/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:23:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:23:53.847736 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6e7b5-feeb-4b2c-bef4-013a718ec40c None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:53.868660 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:53.986771 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6e7b5-feeb-4b2c-bef4-013a718ec40c None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:54.040966 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235ddfc-4b6e-4675-95a1-1dd4ddd32a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:54.041838 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235ddfc-4b6e-4675-95a1-1dd4ddd32a99 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:54.042614 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235ddfc-4b6e-4675-95a1-1dd4ddd32a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:54.042900 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235ddfc-4b6e-4675-95a1-1dd4ddd32a99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:54.058458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235ddfc-4b6e-4675-95a1-1dd4ddd32a99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:54.060248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235ddfc-4b6e-4675-95a1-1dd4ddd32a99 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:54.081791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-d235ddfc-4b6e-4675-95a1-1dd4ddd32a99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:23:54.081791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:23:54.081791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:23:54.081791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:23:54.081791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:23:54.081791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:23:54.081791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:23:54.081791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:23:54.081791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:23:54.081791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:23:54.081791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:23:54.081791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:23:54.081791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:23:54.081791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:23:54.081791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:23:54.082780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:23:54.082780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:23:54.082780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:23:54.082780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:23:54.082780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:23:54.082780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:23:54.082780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:23:54.082780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:23:54.082780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:23:54.082780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:23:54.082780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:23:54.082780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:23:54.082780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:23:54.082780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:23:54.082780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:23:54.082780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:23:54.082780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:23:54.083830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:23:54.083830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:23:54.083830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:23:54.083830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:23:54.088517 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 616/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:23:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:23:54.089487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1541c-15b0-4310-a898-b248b5c2058e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:54.166014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:54.208681 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1541c-15b0-4310-a898-b248b5c2058e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:54.245929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897707b7-06db-464f-82ba-e45e5b738a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:54.246740 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897707b7-06db-464f-82ba-e45e5b738a04 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:54.246918 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897707b7-06db-464f-82ba-e45e5b738a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:54.247207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897707b7-06db-464f-82ba-e45e5b738a04 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:54.247935 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897707b7-06db-464f-82ba-e45e5b738a04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:23:54.248244 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897707b7-06db-464f-82ba-e45e5b738a04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:54.249628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897707b7-06db-464f-82ba-e45e5b738a04 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:54.262103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 600/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:23:54.267832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235ddfc-4b6e-4675-95a1-1dd4ddd32a99 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:54.363848 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:54.396654 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235ddfc-4b6e-4675-95a1-1dd4ddd32a99 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:54.434871 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538deb1-f318-4fb6-afaa-9daed6de9074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:54.435778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538deb1-f318-4fb6-afaa-9daed6de9074 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:54.436184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538deb1-f318-4fb6-afaa-9daed6de9074 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:54.436573 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538deb1-f318-4fb6-afaa-9daed6de9074 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:54.444608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538deb1-f318-4fb6-afaa-9daed6de9074 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:54.446730 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538deb1-f318-4fb6-afaa-9daed6de9074 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:54.461157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-9538deb1-f318-4fb6-afaa-9daed6de9074 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:23:54.461157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:23:54.461157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:23:54.461157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:23:54.461157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:23:54.461157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:23:54.461157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:23:54.461157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:23:54.461157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:23:54.461157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:23:54.461157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:23:54.461157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:23:54.461157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:23:54.461157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:23:54.461157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:23:54.462870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:23:54.462870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:23:54.462870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:23:54.462870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:23:54.462870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:23:54.462870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:23:54.462870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:23:54.462870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:23:54.462870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:23:54.462870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:23:54.462870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:23:54.462870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:23:54.462870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:23:54.462870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:23:54.462870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:23:54.462870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:23:54.462870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:23:54.464014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:23:54.464014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:23:54.464014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:23:54.464014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:23:54.464014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:23:54.464014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:23:54.464014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:23:54.464014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:23:54.465736 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 617/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:23:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:23:54.471371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897707b7-06db-464f-82ba-e45e5b738a04 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:54.582799 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897707b7-06db-464f-82ba-e45e5b738a04 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:54.617268 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:54.638180 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bf5a4-d26c-491a-8c6e-5d08743466fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:54.638180 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bf5a4-d26c-491a-8c6e-5d08743466fb None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:54.638180 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bf5a4-d26c-491a-8c6e-5d08743466fb None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:54.638180 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bf5a4-d26c-491a-8c6e-5d08743466fb None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:54.638658 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bf5a4-d26c-491a-8c6e-5d08743466fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:23:54.638658 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bf5a4-d26c-491a-8c6e-5d08743466fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:54.640100 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bf5a4-d26c-491a-8c6e-5d08743466fb None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:54.664265 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 601/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:23:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:23:54.665575 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538deb1-f318-4fb6-afaa-9daed6de9074 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:54.786605 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538deb1-f318-4fb6-afaa-9daed6de9074 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:54.828806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:54.836830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877aab9-65b3-4dd4-b585-3cca5c55011d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:54.837526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877aab9-65b3-4dd4-b585-3cca5c55011d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:54.837817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877aab9-65b3-4dd4-b585-3cca5c55011d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:54.838845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877aab9-65b3-4dd4-b585-3cca5c55011d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:54.838998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877aab9-65b3-4dd4-b585-3cca5c55011d None admin] RBAC: Adding query filter params (role.id=ea49c36b4a7f4cfb9048065638d458af, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=1afad68819504c2b864851c6989ae25a) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:23:54.839252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877aab9-65b3-4dd4-b585-3cca5c55011d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:54.840673 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877aab9-65b3-4dd4-b585-3cca5c55011d None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:54.848462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 618/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:23:54 2022] GET /identity/v3/role_assignments?user.id=1afad68819504c2b864851c6989ae25a&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=ea49c36b4a7f4cfb9048065638d458af => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:23:57.125668 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e86b1-3788-4975-a473-698a017e5912 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:57.126300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e86b1-3788-4975-a473-698a017e5912 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:57.126300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e86b1-3788-4975-a473-698a017e5912 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:57.127634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 602/1220] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:23:57.134675 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d1c22-2505-4290-9eb0-257ff95eba61 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:57.135200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d1c22-2505-4290-9eb0-257ff95eba61 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:57.135317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d1c22-2505-4290-9eb0-257ff95eba61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:57.153365 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-6d6d1c22-2505-4290-9eb0-257ff95eba61 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:23:57.154740 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6d6d1c22-2505-4290-9eb0-257ff95eba61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:23:57.231981 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 619/1221] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:23:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:23:57.254497 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e86b1-3788-4975-a473-698a017e5912 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:57.378561 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e86b1-3788-4975-a473-698a017e5912 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:57.418002 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff16c6a-1bdc-4d1f-8f35-dead32927888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:57.418792 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff16c6a-1bdc-4d1f-8f35-dead32927888 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:57.419104 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff16c6a-1bdc-4d1f-8f35-dead32927888 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:57.419379 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff16c6a-1bdc-4d1f-8f35-dead32927888 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:57.429730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff16c6a-1bdc-4d1f-8f35-dead32927888 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:23:57.437980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-8ff16c6a-1bdc-4d1f-8f35-dead32927888 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:23:57.437980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:23:57.437980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:23:57.437980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:23:57.437980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:23:57.437980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:23:57.437980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:23:57.437980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:23:57.437980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:23:57.437980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:23:57.437980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:23:57.437980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:23:57.437980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:23:57.437980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:23:57.437980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:23:57.437980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:23:57.439050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:23:57.439050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:23:57.439050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:23:57.439050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:23:57.439050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:23:57.439050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:23:57.439050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:23:57.439050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:23:57.439050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:23:57.439050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:23:57.439050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:23:57.439050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:23:57.439050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:23:57.439050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:23:57.439050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:23:57.439050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:23:57.439996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:23:57.439996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:23:57.439996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:23:57.439996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:23:57.439996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:23:57.439996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:23:57.439996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:23:57.439996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:23:57.439996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:23:57.439996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:23:57.439996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:23:57.439996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:23:57.439996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 603/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:23:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:23:57.445381 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d1c22-2505-4290-9eb0-257ff95eba61 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:57.552044 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d1c22-2505-4290-9eb0-257ff95eba61 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:57.577683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:57.591734 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e9f9d-af38-4cad-bfe2-725566aaf506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:57.592415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e9f9d-af38-4cad-bfe2-725566aaf506 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:57.592707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e9f9d-af38-4cad-bfe2-725566aaf506 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:57.592967 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e9f9d-af38-4cad-bfe2-725566aaf506 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:57.593725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e9f9d-af38-4cad-bfe2-725566aaf506 None admin] RBAC: Adding query filter params (name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:23:57.594041 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e9f9d-af38-4cad-bfe2-725566aaf506 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:57.595493 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e9f9d-af38-4cad-bfe2-725566aaf506 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:57.605235 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 620/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:23:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:23:57.611382 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff16c6a-1bdc-4d1f-8f35-dead32927888 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:57.663326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:57.725579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff16c6a-1bdc-4d1f-8f35-dead32927888 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:57.763564 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772aad5-d9f8-47bb-882b-bfd345b63c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:57.764473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772aad5-d9f8-47bb-882b-bfd345b63c1e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:57.764755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772aad5-d9f8-47bb-882b-bfd345b63c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:57.765064 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772aad5-d9f8-47bb-882b-bfd345b63c1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:57.777830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1772aad5-d9f8-47bb-882b-bfd345b63c1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:57.779450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1772aad5-d9f8-47bb-882b-bfd345b63c1e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:57.797126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-1772aad5-d9f8-47bb-882b-bfd345b63c1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:23:57.797126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:23:57.797126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:23:57.797126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:23:57.797126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:23:57.797126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:23:57.797126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:23:57.797126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:23:57.797126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:23:57.797126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:23:57.797126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:23:57.797126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:23:57.797126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:23:57.797126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:23:57.797126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:23:57.798196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:23:57.798196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:23:57.798196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:23:57.798196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:23:57.798196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:23:57.798196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:23:57.798196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:23:57.798196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:23:57.798196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:23:57.798196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:23:57.798196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:23:57.798196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:23:57.798196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:23:57.798196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:23:57.798196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:23:57.798196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:23:57.798196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:23:57.799137 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:23:57.799137 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:23:57.799137 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:23:57.799137 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:23:57.799137 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 604/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:23:57.804149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e9f9d-af38-4cad-bfe2-725566aaf506 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:57.912842 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e9f9d-af38-4cad-bfe2-725566aaf506 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:57.945482 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56e9f4-286d-4b9d-8a70-ba927724f16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:57.946168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56e9f4-286d-4b9d-8a70-ba927724f16c None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:57.946447 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56e9f4-286d-4b9d-8a70-ba927724f16c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:57.946719 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56e9f4-286d-4b9d-8a70-ba927724f16c None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:57.947639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a56e9f4-286d-4b9d-8a70-ba927724f16c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:23:57.947949 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a56e9f4-286d-4b9d-8a70-ba927724f16c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:57.949713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a56e9f4-286d-4b9d-8a70-ba927724f16c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:57.966734 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 621/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:23:57.973719 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772aad5-d9f8-47bb-882b-bfd345b63c1e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:58.083218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:58.090596 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772aad5-d9f8-47bb-882b-bfd345b63c1e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:58.126333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be1274-0d81-4f16-8e7c-12d54126eca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:58.127166 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be1274-0d81-4f16-8e7c-12d54126eca5 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:58.127445 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be1274-0d81-4f16-8e7c-12d54126eca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:58.127712 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be1274-0d81-4f16-8e7c-12d54126eca5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:58.134859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be1274-0d81-4f16-8e7c-12d54126eca5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:58.136320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be1274-0d81-4f16-8e7c-12d54126eca5 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:58.151353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-98be1274-0d81-4f16-8e7c-12d54126eca5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 20:23:58.151353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:23:58.151353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:23:58.151353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:23:58.151353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:23:58.151353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:23:58.151353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:23:58.151353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:23:58.151353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:23:58.151353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:23:58.151353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:23:58.151353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:23:58.151353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:23:58.151353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:23:58.151353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:23:58.151353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:23:58.152525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:23:58.152525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:23:58.152525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:23:58.152525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:23:58.152525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:23:58.152525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:23:58.152525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:23:58.152525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:23:58.152525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:23:58.152525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:23:58.152525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:23:58.152525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:23:58.152525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:23:58.152525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:23:58.152525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:23:58.152525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:23:58.152525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:23:58.152525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:23:58.153481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:23:58.153481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:23:58.153481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:23:58.153481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:23:58.153481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:23:58.153481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:23:58.153481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:23:58.153481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:23:58.153481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 20:23:58.153481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:23:58.153481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 605/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:23:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:23:58.158917 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56e9f4-286d-4b9d-8a70-ba927724f16c None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:58.209622 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:58.283130 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56e9f4-286d-4b9d-8a70-ba927724f16c None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:58.325028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6adfb-a712-4b43-958e-bb940c02a58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:58.326295 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6adfb-a712-4b43-958e-bb940c02a58e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:58.326644 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6adfb-a712-4b43-958e-bb940c02a58e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:58.326875 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6adfb-a712-4b43-958e-bb940c02a58e None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:58.327952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6adfb-a712-4b43-958e-bb940c02a58e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:23:58.328252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6adfb-a712-4b43-958e-bb940c02a58e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:58.330193 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6adfb-a712-4b43-958e-bb940c02a58e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:58.355632 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 622/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jul 14 20:23:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:23:58.361667 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be1274-0d81-4f16-8e7c-12d54126eca5 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:58.466438 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be1274-0d81-4f16-8e7c-12d54126eca5 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:58.502559 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0d987-5ddf-43bc-8b4a-b85f54b6d9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:58.503381 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0d987-5ddf-43bc-8b4a-b85f54b6d9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:58.503661 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0d987-5ddf-43bc-8b4a-b85f54b6d9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:58.503945 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0d987-5ddf-43bc-8b4a-b85f54b6d9a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:58.515077 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0d987-5ddf-43bc-8b4a-b85f54b6d9a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:58.516601 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0d987-5ddf-43bc-8b4a-b85f54b6d9a3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:58.537085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-42a0d987-5ddf-43bc-8b4a-b85f54b6d9a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:23:58.537085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:23:58.537085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:23:58.537085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:23:58.537085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:23:58.537085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:23:58.537085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:23:58.537085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:23:58.537085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:23:58.537085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:23:58.537085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:23:58.537085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:23:58.537085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:23:58.537085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:23:58.537085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:23:58.538424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:23:58.538424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:23:58.538424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:23:58.538424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:23:58.538424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:23:58.538424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:23:58.538424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:23:58.538424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:23:58.538424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:23:58.538424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:23:58.538424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:23:58.538424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:23:58.538424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:23:58.538424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:23:58.538424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:23:58.538424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:23:58.538424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:23:58.539801 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:23:58.539801 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:23:58.539801 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:23:58.539801 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:23:58.539801 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 606/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:23:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:23:58.544422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6adfb-a712-4b43-958e-bb940c02a58e None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:58.656439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6adfb-a712-4b43-958e-bb940c02a58e None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:58.658950 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:58.693119 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bbbd68-279f-461f-a4cc-5bfb93c644fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:58.693826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bbbd68-279f-461f-a4cc-5bfb93c644fc None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:58.694108 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bbbd68-279f-461f-a4cc-5bfb93c644fc None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:58.694385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bbbd68-279f-461f-a4cc-5bfb93c644fc None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:58.695154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bbbd68-279f-461f-a4cc-5bfb93c644fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:23:58.695463 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bbbd68-279f-461f-a4cc-5bfb93c644fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:58.697652 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bbbd68-279f-461f-a4cc-5bfb93c644fc None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:58.710102 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 623/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:23:58.715639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0d987-5ddf-43bc-8b4a-b85f54b6d9a3 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:58.725759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:58.832289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0d987-5ddf-43bc-8b4a-b85f54b6d9a3 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:58.872746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88838abb-aa1d-4364-83ac-17d9954fe9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:58.873921 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88838abb-aa1d-4364-83ac-17d9954fe9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:58.874346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88838abb-aa1d-4364-83ac-17d9954fe9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:58.874737 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88838abb-aa1d-4364-83ac-17d9954fe9b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:58.882540 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88838abb-aa1d-4364-83ac-17d9954fe9b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:58.885539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88838abb-aa1d-4364-83ac-17d9954fe9b1 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:58.903000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-88838abb-aa1d-4364-83ac-17d9954fe9b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:23:58.903000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:23:58.903000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:23:58.903000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:23:58.903000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:23:58.903000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:23:58.903000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:23:58.903000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:23:58.903000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:23:58.903000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:23:58.903000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:23:58.903000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:23:58.903000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:23:58.903000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:23:58.903000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:23:58.904182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:23:58.904182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:23:58.904182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:23:58.904182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:23:58.904182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:23:58.904182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:23:58.904182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:23:58.904182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:23:58.904182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:23:58.904182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:23:58.904182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:23:58.904182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:23:58.904182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:23:58.904182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:23:58.904182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:23:58.904182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:23:58.904182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:23:58.905256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:23:58.905256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:23:58.905256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:23:58.905256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:23:58.905256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:23:58.905256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:23:58.905256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:23:58.905256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:23:58.905256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 607/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:23:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:23:58.910098 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bbbd68-279f-461f-a4cc-5bfb93c644fc None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:59.022759 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bbbd68-279f-461f-a4cc-5bfb93c644fc None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:59.060093 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32587b5d-6d90-4a16-bdaa-3c4b3361fe02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:59.060806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32587b5d-6d90-4a16-bdaa-3c4b3361fe02 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:59.061112 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32587b5d-6d90-4a16-bdaa-3c4b3361fe02 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:59.061469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32587b5d-6d90-4a16-bdaa-3c4b3361fe02 None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:59.062316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32587b5d-6d90-4a16-bdaa-3c4b3361fe02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:23:59.062627 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32587b5d-6d90-4a16-bdaa-3c4b3361fe02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:59.064441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32587b5d-6d90-4a16-bdaa-3c4b3361fe02 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:59.081075 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 624/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:23:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:23:59.087167 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88838abb-aa1d-4364-83ac-17d9954fe9b1 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:23:59.157789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:23:59.199164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88838abb-aa1d-4364-83ac-17d9954fe9b1 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:23:59.237684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca21b2-36a6-4b7a-bb42-7bdcee63b088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:23:59.238582 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca21b2-36a6-4b7a-bb42-7bdcee63b088 None admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:23:59.238867 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca21b2-36a6-4b7a-bb42-7bdcee63b088 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:23:59.239175 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca21b2-36a6-4b7a-bb42-7bdcee63b088 None admin] PATH_INFO: `/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/1afad68819504c2b864851c6989ae25a/roles/ea49c36b4a7f4cfb9048065638d458af` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:23:59.257113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca21b2-36a6-4b7a-bb42-7bdcee63b088 None admin] RBAC: Authorizing `identity:create_grant(project_id=f33c8ac5154f46a89d1a0dc11fb58cf4, user_id=1afad68819504c2b864851c6989ae25a, role_id=ea49c36b4a7f4cfb9048065638d458af)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:23:59.259254 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca21b2-36a6-4b7a-bb42-7bdcee63b088 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:23:59.277492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 608/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:23:59 2022] PUT /identity/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/1afad68819504c2b864851c6989ae25a/roles/ea49c36b4a7f4cfb9048065638d458af => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:24:01.302124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02868b76-1ac8-4ae9-a4f1-a0c5344924bf None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:01.302820 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02868b76-1ac8-4ae9-a4f1-a0c5344924bf None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:01.302820 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02868b76-1ac8-4ae9-a4f1-a0c5344924bf None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:01.304096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 625/1233] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:24:01.309484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00778f-1c5b-4062-99e3-60c5c3b47b77 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:01.309935 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00778f-1c5b-4062-99e3-60c5c3b47b77 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:01.310053 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00778f-1c5b-4062-99e3-60c5c3b47b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:01.330575 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-fb00778f-1c5b-4062-99e3-60c5c3b47b77 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:24:01.331802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fb00778f-1c5b-4062-99e3-60c5c3b47b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:24:01.406805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 609/1234] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:24:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:24:01.429757 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02868b76-1ac8-4ae9-a4f1-a0c5344924bf None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:01.456404 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:01.545374 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02868b76-1ac8-4ae9-a4f1-a0c5344924bf None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:01.589976 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730f792-b08d-439d-bdf5-63e64b865f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:01.590700 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730f792-b08d-439d-bdf5-63e64b865f2d None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:01.591097 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730f792-b08d-439d-bdf5-63e64b865f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:01.591497 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730f792-b08d-439d-bdf5-63e64b865f2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:01.603237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e730f792-b08d-439d-bdf5-63e64b865f2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:24:01.610886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-e730f792-b08d-439d-bdf5-63e64b865f2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:24:01.610886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:01.610886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:01.610886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:01.610886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:01.610886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:01.610886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:01.610886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:01.610886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:01.610886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:01.610886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:01.610886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:01.610886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:24:01.610886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:24:01.610886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:24:01.610886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:24:01.611872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:24:01.611872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:24:01.611872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:24:01.611872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:24:01.611872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:01.611872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:01.611872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:01.611872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:01.611872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:01.611872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:01.611872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:01.611872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:01.611872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:01.611872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:01.611872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:01.611872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:01.612984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:01.612984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:01.612984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:01.612984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:01.612984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:24:01.612984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:24:01.612984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:24:01.612984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:24:01.612984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:24:01.612984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:24:01.612984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 20:24:01.612984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:24:01.612984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 626/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 14 20:24:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:01.619143 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00778f-1c5b-4062-99e3-60c5c3b47b77 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:01.688846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:01.750819 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00778f-1c5b-4062-99e3-60c5c3b47b77 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:01.791008 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779da1a9-bd17-4ee1-8525-6316f87a6364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:01.791691 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779da1a9-bd17-4ee1-8525-6316f87a6364 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:01.791990 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779da1a9-bd17-4ee1-8525-6316f87a6364 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:01.792252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779da1a9-bd17-4ee1-8525-6316f87a6364 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:01.793017 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779da1a9-bd17-4ee1-8525-6316f87a6364 None admin] RBAC: Adding query filter params (name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:01.793323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779da1a9-bd17-4ee1-8525-6316f87a6364 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:01.794704 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779da1a9-bd17-4ee1-8525-6316f87a6364 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:01.804157 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 610/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 14 20:24:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:01.810309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730f792-b08d-439d-bdf5-63e64b865f2d None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:01.923060 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:01.945198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730f792-b08d-439d-bdf5-63e64b865f2d None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:01.986421 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1e7d9-4664-4589-a685-4cf356ead10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:01.987300 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1e7d9-4664-4589-a685-4cf356ead10e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:01.987758 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1e7d9-4664-4589-a685-4cf356ead10e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:01.988039 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1e7d9-4664-4589-a685-4cf356ead10e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:02.000900 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1e7d9-4664-4589-a685-4cf356ead10e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:02.002372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1e7d9-4664-4589-a685-4cf356ead10e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:02.021797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-6ec1e7d9-4664-4589-a685-4cf356ead10e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:24:02.021797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:02.021797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:02.021797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:02.021797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:02.021797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:02.021797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:02.021797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:02.021797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:02.021797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:02.021797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:02.021797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:02.021797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:24:02.021797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:24:02.021797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:24:02.022861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:24:02.022861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:02.022861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:02.022861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:02.022861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:02.022861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:02.022861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:02.022861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:02.022861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:02.022861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:02.022861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:02.022861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:02.022861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:02.022861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:02.022861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:02.022861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:02.022861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:02.023816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:24:02.023816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:24:02.023816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:24:02.023816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:24:02.023816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 627/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:24:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:02.029034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779da1a9-bd17-4ee1-8525-6316f87a6364 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:02.140110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779da1a9-bd17-4ee1-8525-6316f87a6364 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:02.150855 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:02.182602 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79497484-fa72-4a44-8ef6-bc728f470a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:02.183136 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79497484-fa72-4a44-8ef6-bc728f470a32 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:02.183425 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79497484-fa72-4a44-8ef6-bc728f470a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:02.183706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79497484-fa72-4a44-8ef6-bc728f470a32 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:02.184510 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79497484-fa72-4a44-8ef6-bc728f470a32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:02.184740 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79497484-fa72-4a44-8ef6-bc728f470a32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:02.186120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79497484-fa72-4a44-8ef6-bc728f470a32 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:02.199384 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 611/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:02.205628 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1e7d9-4664-4589-a685-4cf356ead10e None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:02.342546 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1e7d9-4664-4589-a685-4cf356ead10e None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:02.385681 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559ba0a-ab88-448a-996c-149d81fa4cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:02.386490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559ba0a-ab88-448a-996c-149d81fa4cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:02.386821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559ba0a-ab88-448a-996c-149d81fa4cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:02.387146 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559ba0a-ab88-448a-996c-149d81fa4cb0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:02.390480 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:02.396370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559ba0a-ab88-448a-996c-149d81fa4cb0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:02.398140 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559ba0a-ab88-448a-996c-149d81fa4cb0 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:02.420415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-7559ba0a-ab88-448a-996c-149d81fa4cb0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 20:24:02.420415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:02.420415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:02.420415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:02.420415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:02.420415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:02.420415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:02.420415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:02.420415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:02.420415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:02.420415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:02.420415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:02.420415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:24:02.420415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:24:02.420415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:24:02.420415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:24:02.421485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:02.421485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:02.421485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:24:02.421485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:24:02.421485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:24:02.421485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:24:02.421485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:02.421485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:02.421485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:02.421485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:02.421485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:02.421485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:02.421485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:02.421485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:02.421485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:02.421485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:02.421485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:02.421485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:02.422441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:02.422441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:02.422441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:24:02.422441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:24:02.422441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:24:02.422441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:24:02.422441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:24:02.422441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:24:02.422441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 20:24:02.422441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:24:02.422441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 628/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:24:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:02.429665 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79497484-fa72-4a44-8ef6-bc728f470a32 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:02.561670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79497484-fa72-4a44-8ef6-bc728f470a32 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:02.601959 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964c7e9-2f62-4f66-8a49-ceacf735c036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:02.603355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964c7e9-2f62-4f66-8a49-ceacf735c036 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:02.603644 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964c7e9-2f62-4f66-8a49-ceacf735c036 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:02.603931 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964c7e9-2f62-4f66-8a49-ceacf735c036 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:02.605029 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964c7e9-2f62-4f66-8a49-ceacf735c036 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:02.605441 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964c7e9-2f62-4f66-8a49-ceacf735c036 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:02.607183 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964c7e9-2f62-4f66-8a49-ceacf735c036 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:02.627678 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 612/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jul 14 20:24:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:02.633571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559ba0a-ab88-448a-996c-149d81fa4cb0 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:02.749189 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559ba0a-ab88-448a-996c-149d81fa4cb0 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:02.788484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356cf77-ba04-45e6-af55-188a7746b8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:02.789271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356cf77-ba04-45e6-af55-188a7746b8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:02.789560 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356cf77-ba04-45e6-af55-188a7746b8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:02.789962 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356cf77-ba04-45e6-af55-188a7746b8a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:02.866155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:02.875476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356cf77-ba04-45e6-af55-188a7746b8a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:02.877291 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356cf77-ba04-45e6-af55-188a7746b8a0 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:02.999499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 629/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:24:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 20:24:03.005641 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964c7e9-2f62-4f66-8a49-ceacf735c036 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:03.035798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:03.125431 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964c7e9-2f62-4f66-8a49-ceacf735c036 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:03.176578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe0649-7793-46f7-ae62-5af2e4f9a627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:03.177782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe0649-7793-46f7-ae62-5af2e4f9a627 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:03.178211 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe0649-7793-46f7-ae62-5af2e4f9a627 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:03.178610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe0649-7793-46f7-ae62-5af2e4f9a627 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:03.193255 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe0649-7793-46f7-ae62-5af2e4f9a627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:03.194761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe0649-7793-46f7-ae62-5af2e4f9a627 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:03.219577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-bbbe0649-7793-46f7-ae62-5af2e4f9a627 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:24:03.219577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:03.219577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:03.219577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:03.219577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:03.219577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:03.219577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:03.219577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:03.219577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:03.219577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:03.219577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:03.219577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:03.219577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:24:03.219577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:24:03.219577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:24:03.220849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:24:03.220849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:03.220849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:03.220849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:03.220849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:03.220849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:03.220849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:03.220849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:03.220849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:03.220849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:03.220849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:03.220849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:03.220849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:03.220849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:03.220849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:03.220849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:03.220849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:03.221805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:24:03.221805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:24:03.221805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:24:03.221805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:24:03.221805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 613/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:24:03.227369 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356cf77-ba04-45e6-af55-188a7746b8a0 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:03.342653 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:03.351733 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356cf77-ba04-45e6-af55-188a7746b8a0 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:03.401445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efddca-5957-4ff1-be0f-0d4888581f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:03.402052 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efddca-5957-4ff1-be0f-0d4888581f58 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:03.402342 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efddca-5957-4ff1-be0f-0d4888581f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:03.402651 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efddca-5957-4ff1-be0f-0d4888581f58 None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:03.403378 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efddca-5957-4ff1-be0f-0d4888581f58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:03.403731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efddca-5957-4ff1-be0f-0d4888581f58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:03.405302 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efddca-5957-4ff1-be0f-0d4888581f58 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:03.418272 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 630/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:24:03.424075 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe0649-7793-46f7-ae62-5af2e4f9a627 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:03.511847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:03.537524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe0649-7793-46f7-ae62-5af2e4f9a627 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:03.577634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f110d-a558-4dc8-997d-7350fc8d303e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:03.578435 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f110d-a558-4dc8-997d-7350fc8d303e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:03.578713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f110d-a558-4dc8-997d-7350fc8d303e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:03.579012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f110d-a558-4dc8-997d-7350fc8d303e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:03.586181 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f110d-a558-4dc8-997d-7350fc8d303e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:03.587740 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f110d-a558-4dc8-997d-7350fc8d303e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:03.604749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-f08f110d-a558-4dc8-997d-7350fc8d303e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:24:03.604749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:03.604749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:03.604749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:03.604749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:03.604749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:03.604749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:03.604749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:03.604749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:03.604749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:03.604749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:03.604749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:03.604749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:24:03.604749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:24:03.604749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:24:03.605798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:24:03.605798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:03.605798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:03.605798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:03.605798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:03.605798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:03.605798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:03.605798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:03.605798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:03.605798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:03.605798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:03.605798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:03.605798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:03.605798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:03.605798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:03.605798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:03.605798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:03.606746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:24:03.606746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:24:03.606746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:24:03.606746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:24:03.606746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:24:03.606746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:24:03.606746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:24:03.606746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:24:03.606746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 614/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:24:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:03.612329 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efddca-5957-4ff1-be0f-0d4888581f58 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:03.735717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efddca-5957-4ff1-be0f-0d4888581f58 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:03.780235 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b9358-c5ef-4776-acc3-cd450dfb2c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:03.781074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b9358-c5ef-4776-acc3-cd450dfb2c6c None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:03.781402 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b9358-c5ef-4776-acc3-cd450dfb2c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:03.781681 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b9358-c5ef-4776-acc3-cd450dfb2c6c None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:03.782709 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b9358-c5ef-4776-acc3-cd450dfb2c6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:03.783200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b9358-c5ef-4776-acc3-cd450dfb2c6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:03.785447 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b9358-c5ef-4776-acc3-cd450dfb2c6c None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:03.805064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 631/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:24:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:03.806340 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f110d-a558-4dc8-997d-7350fc8d303e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:03.919946 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f110d-a558-4dc8-997d-7350fc8d303e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:03.955544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a3706-e113-4790-9a3f-506fa9c2eb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:03.956398 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a3706-e113-4790-9a3f-506fa9c2eb22 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:03.956714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a3706-e113-4790-9a3f-506fa9c2eb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:03.957019 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a3706-e113-4790-9a3f-506fa9c2eb22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:03.957994 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a3706-e113-4790-9a3f-506fa9c2eb22 None admin] RBAC: Adding query filter params (role.id=ea49c36b4a7f4cfb9048065638d458af, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=1afad68819504c2b864851c6989ae25a) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:03.958405 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a3706-e113-4790-9a3f-506fa9c2eb22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:03.960312 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a3706-e113-4790-9a3f-506fa9c2eb22 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:03.969548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 615/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:24:03 2022] GET /identity/v3/role_assignments?user.id=1afad68819504c2b864851c6989ae25a&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=ea49c36b4a7f4cfb9048065638d458af => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:06.162727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76f1d2-90d4-4bfc-b0ba-bbefb9fe69ee None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:06.163323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76f1d2-90d4-4bfc-b0ba-bbefb9fe69ee None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:06.163452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76f1d2-90d4-4bfc-b0ba-bbefb9fe69ee None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:06.164847 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 632/1247] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:24:06.170697 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7799e-70ce-49ed-821b-5b030980edae None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:06.171238 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7799e-70ce-49ed-821b-5b030980edae None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:06.171356 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7799e-70ce-49ed-821b-5b030980edae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:06.177109 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:06.192494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-13c7799e-70ce-49ed-821b-5b030980edae None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:24:06.193925 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-13c7799e-70ce-49ed-821b-5b030980edae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:24:06.278978 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 616/1248] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:24:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:24:06.298746 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76f1d2-90d4-4bfc-b0ba-bbefb9fe69ee None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:06.323268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:06.446712 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76f1d2-90d4-4bfc-b0ba-bbefb9fe69ee None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:06.487039 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9320a-d2b6-4bd7-a903-ee578540166b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:06.487920 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9320a-d2b6-4bd7-a903-ee578540166b None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:06.488167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9320a-d2b6-4bd7-a903-ee578540166b None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:06.488440 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9320a-d2b6-4bd7-a903-ee578540166b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:06.499682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9320a-d2b6-4bd7-a903-ee578540166b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:24:06.507639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-acd9320a-d2b6-4bd7-a903-ee578540166b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:24:06.507639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:06.507639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:06.507639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:06.507639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:06.507639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:06.507639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:06.507639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:06.507639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:06.507639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:06.507639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:06.507639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:06.507639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:24:06.507639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:24:06.507639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:24:06.507639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:24:06.509357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:24:06.509357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:24:06.509357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:24:06.509357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:24:06.509357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:06.509357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:06.509357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:06.509357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:06.509357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:06.509357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:06.509357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:06.509357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:06.509357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:06.509357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:06.509357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:06.509357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:06.510375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:06.510375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:06.510375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:06.510375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:06.510375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:24:06.510375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:24:06.510375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:24:06.510375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:24:06.510375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:24:06.510375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:24:06.510375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:24:06.510375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:24:06.510375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 633/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:24:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:06.517772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7799e-70ce-49ed-821b-5b030980edae None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:06.641315 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7799e-70ce-49ed-821b-5b030980edae None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:06.677101 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:06.683122 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49a56a-dec3-40d9-bca1-ee51c27e0264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:06.683868 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49a56a-dec3-40d9-bca1-ee51c27e0264 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:06.684229 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49a56a-dec3-40d9-bca1-ee51c27e0264 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:06.684554 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49a56a-dec3-40d9-bca1-ee51c27e0264 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:06.685420 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49a56a-dec3-40d9-bca1-ee51c27e0264 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:06.685729 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49a56a-dec3-40d9-bca1-ee51c27e0264 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:06.687534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49a56a-dec3-40d9-bca1-ee51c27e0264 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:06.698650 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 617/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:24:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:06.704461 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9320a-d2b6-4bd7-a903-ee578540166b None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:06.796184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:06.824169 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9320a-d2b6-4bd7-a903-ee578540166b None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:06.864636 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc260e-b9c4-4435-8965-ca2321c67f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:06.865727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc260e-b9c4-4435-8965-ca2321c67f5f None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:06.866016 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc260e-b9c4-4435-8965-ca2321c67f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:06.866296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc260e-b9c4-4435-8965-ca2321c67f5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:06.879164 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc260e-b9c4-4435-8965-ca2321c67f5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:06.880791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc260e-b9c4-4435-8965-ca2321c67f5f None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:06.899677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-b2dc260e-b9c4-4435-8965-ca2321c67f5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:24:06.899677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:06.899677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:06.899677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:06.899677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:06.899677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:06.899677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:06.899677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:06.899677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:06.899677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:06.899677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:06.899677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:06.899677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:24:06.899677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:24:06.899677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:24:06.900821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:24:06.900821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:06.900821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:06.900821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:06.900821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:06.900821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:06.900821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:06.900821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:06.900821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:06.900821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:06.900821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:06.900821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:06.900821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:06.900821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:06.900821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:06.900821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:06.900821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:06.901825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:24:06.901825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:24:06.901825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:24:06.901825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:24:06.901825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 634/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:24:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:06.907299 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49a56a-dec3-40d9-bca1-ee51c27e0264 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:07.031697 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49a56a-dec3-40d9-bca1-ee51c27e0264 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:07.073292 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d2c78-8565-4206-9f6d-031e511ea83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:07.073974 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d2c78-8565-4206-9f6d-031e511ea83e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:07.074258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d2c78-8565-4206-9f6d-031e511ea83e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:07.074674 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d2c78-8565-4206-9f6d-031e511ea83e None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:07.075854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d2c78-8565-4206-9f6d-031e511ea83e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:07.076166 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d2c78-8565-4206-9f6d-031e511ea83e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:07.077745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d2c78-8565-4206-9f6d-031e511ea83e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:07.093234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 618/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:07.099681 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc260e-b9c4-4435-8965-ca2321c67f5f None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:07.229369 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc260e-b9c4-4435-8965-ca2321c67f5f None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:07.265368 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:07.278203 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92ba30-c1fd-4e45-8bc5-ac2bf37e016a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:07.279350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92ba30-c1fd-4e45-8bc5-ac2bf37e016a None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:07.279809 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92ba30-c1fd-4e45-8bc5-ac2bf37e016a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:07.280273 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92ba30-c1fd-4e45-8bc5-ac2bf37e016a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:07.293860 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92ba30-c1fd-4e45-8bc5-ac2bf37e016a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:07.296211 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92ba30-c1fd-4e45-8bc5-ac2bf37e016a None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:07.314256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-dc92ba30-c1fd-4e45-8bc5-ac2bf37e016a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 20:24:07.314256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:07.314256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:07.314256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:07.314256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:07.314256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:07.314256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:07.314256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:07.314256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:07.314256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:07.314256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:07.314256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:07.314256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:24:07.314256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:24:07.314256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:24:07.314256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:24:07.315998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:07.315998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:07.315998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:24:07.315998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:24:07.315998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:24:07.315998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:24:07.315998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:07.315998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:07.315998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:07.315998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:07.315998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:07.315998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:07.315998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:07.315998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:07.315998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:07.315998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:07.315998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:07.315998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:07.317120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:07.317120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:07.317120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:24:07.317120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:24:07.317120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:24:07.317120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:24:07.317120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:24:07.317120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:24:07.317120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 20:24:07.317120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:24:07.317120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 635/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:24:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:07.323772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d2c78-8565-4206-9f6d-031e511ea83e None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:07.402745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:07.452815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d2c78-8565-4206-9f6d-031e511ea83e None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:07.498598 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c9bfb-0376-435c-b1ea-0717abab49fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:07.499295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c9bfb-0376-435c-b1ea-0717abab49fc None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:07.499577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c9bfb-0376-435c-b1ea-0717abab49fc None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:07.499847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c9bfb-0376-435c-b1ea-0717abab49fc None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:07.500672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c9bfb-0376-435c-b1ea-0717abab49fc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:07.500976 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c9bfb-0376-435c-b1ea-0717abab49fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:07.502392 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c9bfb-0376-435c-b1ea-0717abab49fc None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:07.528755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 619/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jul 14 20:24:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:07.530326 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92ba30-c1fd-4e45-8bc5-ac2bf37e016a None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:07.652585 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92ba30-c1fd-4e45-8bc5-ac2bf37e016a None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:07.692524 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d9052-a194-47ed-96d1-141b4ee529e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:07.693570 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d9052-a194-47ed-96d1-141b4ee529e4 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:07.693570 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d9052-a194-47ed-96d1-141b4ee529e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:07.693775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d9052-a194-47ed-96d1-141b4ee529e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:07.763209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:07.778697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d9052-a194-47ed-96d1-141b4ee529e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:07.780203 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d9052-a194-47ed-96d1-141b4ee529e4 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:07.900033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 636/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:24:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:24:07.906108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c9bfb-0376-435c-b1ea-0717abab49fc None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:08.027513 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c9bfb-0376-435c-b1ea-0717abab49fc None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:08.043632 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:08.068871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e790e-578a-42f6-912c-5af429461ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:08.069812 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e790e-578a-42f6-912c-5af429461ade None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:08.070092 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e790e-578a-42f6-912c-5af429461ade None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:08.070466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e790e-578a-42f6-912c-5af429461ade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:08.082022 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e790e-578a-42f6-912c-5af429461ade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:08.083839 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e790e-578a-42f6-912c-5af429461ade None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:08.105245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-2f4e790e-578a-42f6-912c-5af429461ade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:24:08.105245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:08.105245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:08.105245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:08.105245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:08.105245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:08.105245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:08.105245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:08.105245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:08.105245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:08.105245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:08.105245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:08.105245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:24:08.105245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:24:08.105245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:24:08.106605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:24:08.106605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:08.106605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:08.106605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:08.106605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:08.106605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:08.106605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:08.106605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:08.106605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:08.106605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:08.106605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:08.106605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:08.106605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:08.106605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:08.106605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:08.106605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:08.106605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:08.107735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:24:08.107735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:24:08.107735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:24:08.107735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:24:08.107735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 620/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:24:08.114143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d9052-a194-47ed-96d1-141b4ee529e4 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:08.209176 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:08.222532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d9052-a194-47ed-96d1-141b4ee529e4 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:08.267671 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed84a0-dead-4d00-bd4e-9606cd3fae2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:08.268643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed84a0-dead-4d00-bd4e-9606cd3fae2f None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:08.268937 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed84a0-dead-4d00-bd4e-9606cd3fae2f None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:08.269253 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed84a0-dead-4d00-bd4e-9606cd3fae2f None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:08.270195 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed84a0-dead-4d00-bd4e-9606cd3fae2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:08.270544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed84a0-dead-4d00-bd4e-9606cd3fae2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:08.272216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed84a0-dead-4d00-bd4e-9606cd3fae2f None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:08.286639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 637/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:08.293888 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e790e-578a-42f6-912c-5af429461ade None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:08.426282 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e790e-578a-42f6-912c-5af429461ade None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:08.465526 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff692b-28d8-447d-9270-c48c152b0426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:08.466230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff692b-28d8-447d-9270-c48c152b0426 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:08.466521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff692b-28d8-447d-9270-c48c152b0426 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:08.466745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff692b-28d8-447d-9270-c48c152b0426 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:08.474132 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff692b-28d8-447d-9270-c48c152b0426 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:08.475620 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff692b-28d8-447d-9270-c48c152b0426 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:08.490847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-b6ff692b-28d8-447d-9270-c48c152b0426 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:24:08.490847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:08.490847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:08.490847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:08.490847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:08.490847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:08.490847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:08.490847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:08.490847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:08.490847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:08.490847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:08.490847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:08.490847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:24:08.490847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:24:08.490847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:24:08.492000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:24:08.492000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:08.492000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:08.492000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:08.492000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:08.492000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:08.492000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:08.492000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:08.492000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:08.492000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:08.492000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:08.492000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:08.492000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:08.492000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:08.492000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:08.492000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:08.492000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:08.493048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:24:08.493048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:24:08.493048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:24:08.493048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:24:08.493048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:24:08.493048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:24:08.493048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:24:08.493048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:24:08.493048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 621/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:24:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:08.498524 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed84a0-dead-4d00-bd4e-9606cd3fae2f None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:08.620865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed84a0-dead-4d00-bd4e-9606cd3fae2f None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:08.661570 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a72807-7d11-4769-85df-af28f1b0dd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:08.662315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a72807-7d11-4769-85df-af28f1b0dd82 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:08.662602 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a72807-7d11-4769-85df-af28f1b0dd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:08.662875 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a72807-7d11-4769-85df-af28f1b0dd82 None admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:08.663709 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a72807-7d11-4769-85df-af28f1b0dd82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:08.664019 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a72807-7d11-4769-85df-af28f1b0dd82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:08.668426 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a72807-7d11-4769-85df-af28f1b0dd82 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:08.683190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 638/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:24:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:08.692783 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff692b-28d8-447d-9270-c48c152b0426 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:08.721300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:08.806786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff692b-28d8-447d-9270-c48c152b0426 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:08.844370 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d425f-1320-4f1b-8b6a-0b25511c121e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:08.845305 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d425f-1320-4f1b-8b6a-0b25511c121e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:08.845727 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d425f-1320-4f1b-8b6a-0b25511c121e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:08.846120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d425f-1320-4f1b-8b6a-0b25511c121e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:08.847418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d425f-1320-4f1b-8b6a-0b25511c121e None admin] RBAC: Adding query filter params (role.id=5ca112885467422684e134cb0f94865a, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=1afad68819504c2b864851c6989ae25a) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:08.847732 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d425f-1320-4f1b-8b6a-0b25511c121e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:08.849719 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d425f-1320-4f1b-8b6a-0b25511c121e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:08.856751 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 622/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:24:08 2022] GET /identity/v3/role_assignments?user.id=1afad68819504c2b864851c6989ae25a&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=5ca112885467422684e134cb0f94865a => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:10.908407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fba3f6-a14e-4542-8b2a-6330b1a7d7ac None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:10.908923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fba3f6-a14e-4542-8b2a-6330b1a7d7ac None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:10.908923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fba3f6-a14e-4542-8b2a-6330b1a7d7ac None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:10.910381 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 639/1261] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:24:10.916466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25142597-6947-4fd9-b53a-cdcbe0c093f9 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:10.916918 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25142597-6947-4fd9-b53a-cdcbe0c093f9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:10.917009 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25142597-6947-4fd9-b53a-cdcbe0c093f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:10.933963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-25142597-6947-4fd9-b53a-cdcbe0c093f9 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:24:10.935168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-25142597-6947-4fd9-b53a-cdcbe0c093f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:24:11.000118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 623/1262] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:24:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:24:11.021865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fba3f6-a14e-4542-8b2a-6330b1a7d7ac None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:11.035081 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:11.124494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fba3f6-a14e-4542-8b2a-6330b1a7d7ac None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:11.156532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3b690-1b47-4053-bd3a-42b7ebbc1ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:11.157305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3b690-1b47-4053-bd3a-42b7ebbc1ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:11.157584 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3b690-1b47-4053-bd3a-42b7ebbc1ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:11.157937 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3b690-1b47-4053-bd3a-42b7ebbc1ac4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:11.168211 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3b690-1b47-4053-bd3a-42b7ebbc1ac4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:24:11.175540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-f7e3b690-1b47-4053-bd3a-42b7ebbc1ac4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:24:11.175540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:11.175540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:11.175540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:11.175540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:11.175540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:11.175540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:11.175540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:11.175540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:11.175540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:11.175540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:11.175540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:11.175540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:24:11.175540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:24:11.175540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:24:11.175540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:24:11.176625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:24:11.176625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:24:11.176625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:24:11.176625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:24:11.176625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:11.176625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:11.176625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:11.176625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:11.176625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:11.176625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:11.176625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:11.176625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:11.176625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:11.176625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:11.176625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:11.176625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:11.183026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:11.183026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:11.183026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:11.183026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:11.183026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:24:11.183026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:24:11.183026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:24:11.183026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:24:11.183026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:24:11.183026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:24:11.183026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:24:11.183026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:24:11.183026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 640/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:24:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 20:24:11.183886 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25142597-6947-4fd9-b53a-cdcbe0c093f9 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:11.201174 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:11.282793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25142597-6947-4fd9-b53a-cdcbe0c093f9 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:11.316973 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017618d4-69cc-4a10-970d-3d2cca7beaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:11.317721 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017618d4-69cc-4a10-970d-3d2cca7beaf6 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:11.318003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017618d4-69cc-4a10-970d-3d2cca7beaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:11.318276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017618d4-69cc-4a10-970d-3d2cca7beaf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:11.319042 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017618d4-69cc-4a10-970d-3d2cca7beaf6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:11.319348 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017618d4-69cc-4a10-970d-3d2cca7beaf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:11.320711 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017618d4-69cc-4a10-970d-3d2cca7beaf6 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:11.333258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 624/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:24:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:11.340385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3b690-1b47-4053-bd3a-42b7ebbc1ac4 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:11.416389 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:11.449122 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3b690-1b47-4053-bd3a-42b7ebbc1ac4 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:11.481673 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a7dd3-8805-4f89-a338-8153540f4d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:11.482478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a7dd3-8805-4f89-a338-8153540f4d76 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:11.482760 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a7dd3-8805-4f89-a338-8153540f4d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:11.483062 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a7dd3-8805-4f89-a338-8153540f4d76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:11.493190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a7dd3-8805-4f89-a338-8153540f4d76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:11.494633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a7dd3-8805-4f89-a338-8153540f4d76 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:11.511132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-de6a7dd3-8805-4f89-a338-8153540f4d76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:24:11.511132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:11.511132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:11.511132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:11.511132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:11.511132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:11.511132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:11.511132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:11.511132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:11.511132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:11.511132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:11.511132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:11.511132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:24:11.511132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:24:11.511132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:24:11.512184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:24:11.512184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:11.512184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:11.512184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:11.512184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:11.512184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:11.512184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:11.512184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:11.512184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:11.512184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:11.512184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:11.512184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:11.512184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:11.512184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:11.512184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:11.512184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:11.512184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:11.513282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:24:11.513282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:24:11.513282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:24:11.513282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:24:11.513282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 641/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:11.518459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017618d4-69cc-4a10-970d-3d2cca7beaf6 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:11.597725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:11.616758 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017618d4-69cc-4a10-970d-3d2cca7beaf6 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:11.650414 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c2ca2-fec9-4d5a-a843-eb747db8605d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:11.651109 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c2ca2-fec9-4d5a-a843-eb747db8605d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:11.651393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c2ca2-fec9-4d5a-a843-eb747db8605d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:11.651667 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c2ca2-fec9-4d5a-a843-eb747db8605d None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:11.652376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c2ca2-fec9-4d5a-a843-eb747db8605d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:11.652679 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c2ca2-fec9-4d5a-a843-eb747db8605d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:11.654035 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c2ca2-fec9-4d5a-a843-eb747db8605d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:11.666474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 625/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:11.672167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a7dd3-8805-4f89-a338-8153540f4d76 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:11.773670 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a7dd3-8805-4f89-a338-8153540f4d76 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:11.793468 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:11.807776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951f3af-9749-4449-8b40-ae8afec31694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:11.808603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951f3af-9749-4449-8b40-ae8afec31694 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:11.808885 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951f3af-9749-4449-8b40-ae8afec31694 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:11.809159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951f3af-9749-4449-8b40-ae8afec31694 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:11.816750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951f3af-9749-4449-8b40-ae8afec31694 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:11.818177 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951f3af-9749-4449-8b40-ae8afec31694 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:11.832639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-5951f3af-9749-4449-8b40-ae8afec31694 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 20:24:11.832639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:11.832639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:11.832639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:11.832639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:11.832639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:11.832639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:11.832639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:11.832639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:11.832639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:11.832639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:11.832639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:11.832639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:24:11.832639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:24:11.832639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:24:11.832639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:24:11.833762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:11.833762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:11.833762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:24:11.833762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:24:11.833762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:24:11.833762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:24:11.833762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:11.833762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:11.833762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:11.833762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:11.833762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:11.833762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:11.833762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:11.833762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:11.833762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:11.833762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:11.833762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:11.833762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:11.834960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:11.834960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:11.834960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:24:11.834960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:24:11.834960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:24:11.834960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:24:11.834960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:24:11.834960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:24:11.834960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 20:24:11.834960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:24:11.834960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 642/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:24:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:11.840289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c2ca2-fec9-4d5a-a843-eb747db8605d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:11.939766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c2ca2-fec9-4d5a-a843-eb747db8605d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:11.972907 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7455e1b-d87b-4519-b52a-943ce7fdffd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:11.973559 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7455e1b-d87b-4519-b52a-943ce7fdffd0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:11.973838 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7455e1b-d87b-4519-b52a-943ce7fdffd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:11.974108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7455e1b-d87b-4519-b52a-943ce7fdffd0 None admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:11.974923 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7455e1b-d87b-4519-b52a-943ce7fdffd0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:11.975253 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7455e1b-d87b-4519-b52a-943ce7fdffd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:11.976656 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7455e1b-d87b-4519-b52a-943ce7fdffd0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:11.985280 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:11.994363 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 626/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jul 14 20:24:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:24:12.000230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951f3af-9749-4449-8b40-ae8afec31694 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:12.100905 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951f3af-9749-4449-8b40-ae8afec31694 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:12.144252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce5395-844f-48d3-94b9-aec07702c5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:12.145055 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce5395-844f-48d3-94b9-aec07702c5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:12.145340 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce5395-844f-48d3-94b9-aec07702c5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:12.145617 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce5395-844f-48d3-94b9-aec07702c5d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:12.156439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce5395-844f-48d3-94b9-aec07702c5d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:12.157890 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce5395-844f-48d3-94b9-aec07702c5d5 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:12.174284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-18ce5395-844f-48d3-94b9-aec07702c5d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:24:12.174284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:12.174284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:12.174284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:12.174284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:12.174284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:12.174284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:12.174284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:12.174284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:12.174284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:12.174284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:12.174284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:12.174284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:24:12.174284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:24:12.174284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:24:12.175350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:24:12.175350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:12.175350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:12.175350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:12.175350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:12.175350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:12.175350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:12.175350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:12.175350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:12.175350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:12.175350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:12.175350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:12.175350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:12.175350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:12.175350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:12.175350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:12.175350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:12.176444 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:24:12.176444 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:24:12.176444 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:24:12.176444 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:24:12.176444 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 643/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:12.182053 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7455e1b-d87b-4519-b52a-943ce7fdffd0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:12.282082 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7455e1b-d87b-4519-b52a-943ce7fdffd0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:12.319550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f014fa-f297-4d5c-8a67-71c639852bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:12.320415 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f014fa-f297-4d5c-8a67-71c639852bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:12.320711 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f014fa-f297-4d5c-8a67-71c639852bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:12.321101 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f014fa-f297-4d5c-8a67-71c639852bc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:12.323130 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f014fa-f297-4d5c-8a67-71c639852bc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:12.323611 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f014fa-f297-4d5c-8a67-71c639852bc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:12.325896 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f014fa-f297-4d5c-8a67-71c639852bc0 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:12.340083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 627/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:12.346791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce5395-844f-48d3-94b9-aec07702c5d5 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:12.385695 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:12.459136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce5395-844f-48d3-94b9-aec07702c5d5 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:12.493007 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08422cd-88ed-4686-9c7b-3b9537884684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:12.494363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08422cd-88ed-4686-9c7b-3b9537884684 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:12.494636 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08422cd-88ed-4686-9c7b-3b9537884684 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:12.494903 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08422cd-88ed-4686-9c7b-3b9537884684 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:12.501538 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08422cd-88ed-4686-9c7b-3b9537884684 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:12.502985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08422cd-88ed-4686-9c7b-3b9537884684 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:12.516749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-e08422cd-88ed-4686-9c7b-3b9537884684 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:24:12.516749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:12.516749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:12.516749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:12.516749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:12.516749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:12.516749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:12.516749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:12.516749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:12.516749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:12.516749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:12.516749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:12.516749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:24:12.516749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:24:12.516749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:24:12.518454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:24:12.518454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:12.518454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:12.518454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:12.518454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:12.518454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:12.518454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:12.518454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:12.518454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:12.518454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:12.518454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:12.518454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:12.518454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:12.518454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:12.518454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:12.518454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:12.518454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:12.520171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:24:12.520171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:24:12.520171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:24:12.520171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:24:12.520171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:24:12.520171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:24:12.520171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:24:12.520171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:24:12.520171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 644/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:24:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:12.524020 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f014fa-f297-4d5c-8a67-71c639852bc0 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:12.581065 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:12.622960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f014fa-f297-4d5c-8a67-71c639852bc0 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:12.656830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ff699-2697-4edc-b661-39dc4b0c485e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:12.657412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ff699-2697-4edc-b661-39dc4b0c485e None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:12.657697 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ff699-2697-4edc-b661-39dc4b0c485e None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:12.657970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ff699-2697-4edc-b661-39dc4b0c485e None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:12.658760 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ff699-2697-4edc-b661-39dc4b0c485e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:12.659102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ff699-2697-4edc-b661-39dc4b0c485e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:12.660533 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ff699-2697-4edc-b661-39dc4b0c485e None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:12.673083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 628/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:24:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:12.679187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08422cd-88ed-4686-9c7b-3b9537884684 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:12.770261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:12.779550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08422cd-88ed-4686-9c7b-3b9537884684 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:12.812527 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adee861-91c1-4864-ac17-b1b1bbb6278a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:12.813410 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adee861-91c1-4864-ac17-b1b1bbb6278a None admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:12.813690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adee861-91c1-4864-ac17-b1b1bbb6278a None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:12.813963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adee861-91c1-4864-ac17-b1b1bbb6278a None admin] PATH_INFO: `/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/1afad68819504c2b864851c6989ae25a/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:12.819034 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adee861-91c1-4864-ac17-b1b1bbb6278a None admin] RBAC: Authorizing `identity:create_grant(project_id=f33c8ac5154f46a89d1a0dc11fb58cf4, user_id=1afad68819504c2b864851c6989ae25a, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:12.820466 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adee861-91c1-4864-ac17-b1b1bbb6278a None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:12.835296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 645/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 14 20:24:12 2022] PUT /identity/v3/projects/f33c8ac5154f46a89d1a0dc11fb58cf4/users/1afad68819504c2b864851c6989ae25a/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:24:14.834233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ff65a-58be-42c8-b83c-db8191fc2dd9 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:14.834814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ff65a-58be-42c8-b83c-db8191fc2dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:14.834814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ff65a-58be-42c8-b83c-db8191fc2dd9 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:14.836269 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 629/1274] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:24:14.842323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bb8ee-20bf-4bee-af87-2594ae3dd147 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:14.842845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bb8ee-20bf-4bee-af87-2594ae3dd147 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:14.842953 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bb8ee-20bf-4bee-af87-2594ae3dd147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:14.861238 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-471bb8ee-20bf-4bee-af87-2594ae3dd147 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:24:14.862466 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-471bb8ee-20bf-4bee-af87-2594ae3dd147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:24:14.934929 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 646/1275] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:24:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:24:14.957892 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ff65a-58be-42c8-b83c-db8191fc2dd9 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:15.060161 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ff65a-58be-42c8-b83c-db8191fc2dd9 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:15.078094 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:15.095562 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b0afa-fb45-4ea3-be5b-ee8bf9bc3323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:15.096348 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b0afa-fb45-4ea3-be5b-ee8bf9bc3323 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:15.096631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b0afa-fb45-4ea3-be5b-ee8bf9bc3323 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:15.096907 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b0afa-fb45-4ea3-be5b-ee8bf9bc3323 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:15.107658 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b0afa-fb45-4ea3-be5b-ee8bf9bc3323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 20:24:15.115263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-8a8b0afa-fb45-4ea3-be5b-ee8bf9bc3323 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:24:15.115263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:15.115263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:15.115263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:15.115263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:15.115263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:15.115263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:15.115263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:15.115263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:15.115263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:15.115263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:15.115263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:15.115263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 20:24:15.115263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 20:24:15.115263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 20:24:15.115263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 20:24:15.116359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 20:24:15.116359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 20:24:15.116359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 20:24:15.116359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 20:24:15.116359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:15.116359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:15.116359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:15.116359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:15.116359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:15.116359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:15.116359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:15.116359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:15.116359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:15.116359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:15.116359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:15.116359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:15.117286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:15.117286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:15.117286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:15.117286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:15.117286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 20:24:15.117286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 20:24:15.117286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 20:24:15.117286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 20:24:15.117286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 20:24:15.117286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 20:24:15.117286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 20:24:15.117286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:24:15.117286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 630/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 14 20:24:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:15.122479 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bb8ee-20bf-4bee-af87-2594ae3dd147 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:15.203009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:15.222257 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bb8ee-20bf-4bee-af87-2594ae3dd147 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:15.258681 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04641c7-f540-4c8d-8f2f-229390006ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:15.259409 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04641c7-f540-4c8d-8f2f-229390006ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:15.259697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04641c7-f540-4c8d-8f2f-229390006ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:15.259977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04641c7-f540-4c8d-8f2f-229390006ec4 None admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:15.260731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04641c7-f540-4c8d-8f2f-229390006ec4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:15.261044 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04641c7-f540-4c8d-8f2f-229390006ec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:15.262413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04641c7-f540-4c8d-8f2f-229390006ec4 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:15.271550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 647/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 14 20:24:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:15.277895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b0afa-fb45-4ea3-be5b-ee8bf9bc3323 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:15.380014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b0afa-fb45-4ea3-be5b-ee8bf9bc3323 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:15.413525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3c058-dcd9-47b1-b411-d4691d3c29ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:15.414316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3c058-dcd9-47b1-b411-d4691d3c29ca None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:15.414596 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3c058-dcd9-47b1-b411-d4691d3c29ca None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:15.414866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3c058-dcd9-47b1-b411-d4691d3c29ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:15.425214 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3c058-dcd9-47b1-b411-d4691d3c29ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:15.426645 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3c058-dcd9-47b1-b411-d4691d3c29ca None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:15.443306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-c6b3c058-dcd9-47b1-b411-d4691d3c29ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:24:15.443306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:15.443306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:15.443306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:15.443306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:15.443306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:15.443306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:15.443306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:15.443306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:15.443306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:15.443306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:15.443306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:15.443306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:24:15.443306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:24:15.443306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:24:15.444164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:24:15.444164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:15.444164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:15.444164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:15.444164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:15.444164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:15.444164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:15.444164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:15.444164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:15.444164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:15.444164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:15.444164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:15.444164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:15.444164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:15.444164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:15.444164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:15.444164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:15.445072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:24:15.445072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:24:15.445072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:24:15.445072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:24:15.445072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 631/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:15.450838 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04641c7-f540-4c8d-8f2f-229390006ec4 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:15.550363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04641c7-f540-4c8d-8f2f-229390006ec4 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:15.583432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240fecc7-bab9-4a3a-a1db-c6ee5791b880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:15.584092 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240fecc7-bab9-4a3a-a1db-c6ee5791b880 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:15.584377 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240fecc7-bab9-4a3a-a1db-c6ee5791b880 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:15.584654 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240fecc7-bab9-4a3a-a1db-c6ee5791b880 None admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:15.585375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240fecc7-bab9-4a3a-a1db-c6ee5791b880 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:15.585683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240fecc7-bab9-4a3a-a1db-c6ee5791b880 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:15.587637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240fecc7-bab9-4a3a-a1db-c6ee5791b880 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:15.599142 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 648/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:15.604965 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3c058-dcd9-47b1-b411-d4691d3c29ca None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:15.636197 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:15.705930 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3c058-dcd9-47b1-b411-d4691d3c29ca None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:15.739594 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8a7d3-2e8f-4fce-8f53-e208c4e17958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:15.740388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8a7d3-2e8f-4fce-8f53-e208c4e17958 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:15.740670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8a7d3-2e8f-4fce-8f53-e208c4e17958 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:15.740954 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8a7d3-2e8f-4fce-8f53-e208c4e17958 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:15.749291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8a7d3-2e8f-4fce-8f53-e208c4e17958 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:15.750740 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8a7d3-2e8f-4fce-8f53-e208c4e17958 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:15.765385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-dda8a7d3-2e8f-4fce-8f53-e208c4e17958 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 20:24:15.765385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:15.765385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:15.765385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:15.765385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:15.765385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:15.765385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:15.765385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:15.765385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:15.765385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:15.765385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:15.765385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:15.765385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 20:24:15.765385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 20:24:15.765385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 20:24:15.765385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 20:24:15.766578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:15.766578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:15.766578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 20:24:15.766578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:24:15.766578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 20:24:15.766578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 20:24:15.766578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:15.766578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:15.766578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:15.766578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:15.766578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:15.766578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:15.766578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:15.766578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:15.766578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:15.766578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:15.766578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:15.766578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:15.768001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:15.768001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:15.768001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 20:24:15.768001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 20:24:15.768001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 20:24:15.768001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 20:24:15.768001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 20:24:15.768001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 20:24:15.768001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 20:24:15.768001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:24:15.768001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 632/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:24:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:15.772699 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240fecc7-bab9-4a3a-a1db-c6ee5791b880 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:15.776546 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:15.869700 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240fecc7-bab9-4a3a-a1db-c6ee5791b880 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:15.906585 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b634019-6c9d-4211-8798-40caaee59808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:15.907281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b634019-6c9d-4211-8798-40caaee59808 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:15.907559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b634019-6c9d-4211-8798-40caaee59808 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:15.907827 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b634019-6c9d-4211-8798-40caaee59808 None admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:15.908655 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b634019-6c9d-4211-8798-40caaee59808 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:15.908959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b634019-6c9d-4211-8798-40caaee59808 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:15.910377 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b634019-6c9d-4211-8798-40caaee59808 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:15.927455 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 649/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jul 14 20:24:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:15.933874 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8a7d3-2e8f-4fce-8f53-e208c4e17958 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:16.022740 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:16.032666 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8a7d3-2e8f-4fce-8f53-e208c4e17958 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:16.067167 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07777d-707b-4e7a-8a12-efe253a89624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:16.068135 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07777d-707b-4e7a-8a12-efe253a89624 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:16.068413 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07777d-707b-4e7a-8a12-efe253a89624 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:16.068687 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07777d-707b-4e7a-8a12-efe253a89624 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:16.134604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07777d-707b-4e7a-8a12-efe253a89624 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:16.136020 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07777d-707b-4e7a-8a12-efe253a89624 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:16.222553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:16.234165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 633/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 14 20:24:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 20:24:16.240133 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b634019-6c9d-4211-8798-40caaee59808 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:16.301223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:16.348308 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b634019-6c9d-4211-8798-40caaee59808 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:16.384200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ad6e4-e5fb-47f1-9b0e-b7218bf130db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:16.385056 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ad6e4-e5fb-47f1-9b0e-b7218bf130db None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:16.385282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ad6e4-e5fb-47f1-9b0e-b7218bf130db None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:16.385557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ad6e4-e5fb-47f1-9b0e-b7218bf130db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:16.396061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ad6e4-e5fb-47f1-9b0e-b7218bf130db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:16.397515 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ad6e4-e5fb-47f1-9b0e-b7218bf130db None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:16.414696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-bf6ad6e4-e5fb-47f1-9b0e-b7218bf130db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:24:16.414696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:16.414696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:16.414696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:16.414696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:16.414696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:16.414696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:16.414696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:16.414696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:16.414696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:16.414696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:16.414696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:16.414696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 20:24:16.414696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 20:24:16.414696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 20:24:16.416315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 20:24:16.416315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:16.416315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:16.416315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:16.416315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:16.416315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:16.416315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:16.416315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:16.416315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:16.416315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:16.416315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:16.416315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:16.416315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:16.416315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:16.416315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:16.416315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:16.416315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:16.417416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 20:24:16.417416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 20:24:16.417416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 20:24:16.417416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:24:16.417416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 650/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:16.422228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07777d-707b-4e7a-8a12-efe253a89624 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:16.518675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07777d-707b-4e7a-8a12-efe253a89624 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:16.552174 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7bd7c4-700f-4ebf-8755-949db161f1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:16.552828 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7bd7c4-700f-4ebf-8755-949db161f1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:16.553112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7bd7c4-700f-4ebf-8755-949db161f1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:16.553387 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7bd7c4-700f-4ebf-8755-949db161f1b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:16.554107 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7bd7c4-700f-4ebf-8755-949db161f1b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:16.554413 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7bd7c4-700f-4ebf-8755-949db161f1b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:16.555857 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7bd7c4-700f-4ebf-8755-949db161f1b7 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:16.567028 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 634/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 14 20:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:16.572688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ad6e4-e5fb-47f1-9b0e-b7218bf130db None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:16.672464 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ad6e4-e5fb-47f1-9b0e-b7218bf130db None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:16.686467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:16.707317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ada8e-6b1f-4a25-a7e4-575271cc694e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:16.708113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ada8e-6b1f-4a25-a7e4-575271cc694e None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:16.708397 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ada8e-6b1f-4a25-a7e4-575271cc694e None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:16.708674 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ada8e-6b1f-4a25-a7e4-575271cc694e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:16.715903 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ada8e-6b1f-4a25-a7e4-575271cc694e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:16.717341 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ada8e-6b1f-4a25-a7e4-575271cc694e None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:16.731306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application [None req-2e7ada8e-6b1f-4a25-a7e4-575271cc694e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:24:16.731306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:16.731306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:16.731306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:16.731306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:16.731306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:16.731306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:16.731306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:16.731306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:16.731306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:16.731306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:16.731306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:16.731306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 20:24:16.731306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 20:24:16.731306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 20:24:16.732391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 20:24:16.732391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:16.732391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:16.732391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:16.732391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:16.732391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:16.732391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:16.732391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:16.732391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:16.732391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:16.732391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:16.732391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:16.732391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:16.732391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:16.732391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:16.732391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:16.732391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:16.733580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 20:24:16.733580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 20:24:16.733580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 20:24:16.733580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 20:24:16.733580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 20:24:16.733580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 20:24:16.733580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 20:24:16.733580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: ERROR keystone.server.flask.application Jul 14 20:24:16.733580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 651/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 14 20:24:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:16.738372 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7bd7c4-700f-4ebf-8755-949db161f1b7 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:16.788297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:16.839889 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7bd7c4-700f-4ebf-8755-949db161f1b7 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:16.873331 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac085954-232b-4f05-9c04-597e28a19ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:16.873988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac085954-232b-4f05-9c04-597e28a19ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:16.874272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac085954-232b-4f05-9c04-597e28a19ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:16.874543 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac085954-232b-4f05-9c04-597e28a19ee3 None admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:16.875357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac085954-232b-4f05-9c04-597e28a19ee3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:16.875663 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac085954-232b-4f05-9c04-597e28a19ee3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:16.877090 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac085954-232b-4f05-9c04-597e28a19ee3 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:16.890305 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 635/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 14 20:24:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:16.896980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ada8e-6b1f-4a25-a7e4-575271cc694e None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:16.997129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ada8e-6b1f-4a25-a7e4-575271cc694e None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:17.030813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35da292-e470-4535-81e8-6f9e3af81c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:17.031526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35da292-e470-4535-81e8-6f9e3af81c22 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:17.031813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35da292-e470-4535-81e8-6f9e3af81c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:17.032090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35da292-e470-4535-81e8-6f9e3af81c22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:17.032924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35da292-e470-4535-81e8-6f9e3af81c22 None admin] RBAC: Adding query filter params (role.id=5ca112885467422684e134cb0f94865a, scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4, user.id=1afad68819504c2b864851c6989ae25a) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:17.033236 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35da292-e470-4535-81e8-6f9e3af81c22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:17.034655 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35da292-e470-4535-81e8-6f9e3af81c22 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:17.040532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 652/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 14 20:24:16 2022] GET /identity/v3/role_assignments?user.id=1afad68819504c2b864851c6989ae25a&scope.project.id=f33c8ac5154f46a89d1a0dc11fb58cf4&role.id=5ca112885467422684e134cb0f94865a => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:19.088100 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748467a-6131-4c05-83cd-c636cd82a5ee None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:19.088664 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748467a-6131-4c05-83cd-c636cd82a5ee None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:19.088664 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748467a-6131-4c05-83cd-c636cd82a5ee None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:19.090057 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 636/1288] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:24:19.095058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618cafb-8928-46fc-8993-605624044b73 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:19.095500 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618cafb-8928-46fc-8993-605624044b73 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:19.095604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618cafb-8928-46fc-8993-605624044b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:19.111954 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-2618cafb-8928-46fc-8993-605624044b73 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:24:19.113123 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-2618cafb-8928-46fc-8993-605624044b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:24:19.142678 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:19.179441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 653/1289] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:24:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:24:19.201812 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748467a-6131-4c05-83cd-c636cd82a5ee None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:19.300624 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748467a-6131-4c05-83cd-c636cd82a5ee None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:19.314165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:19.344730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b019e3-df65-43c2-a847-698064ae402d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:19.345983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b019e3-df65-43c2-a847-698064ae402d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:19.346773 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b019e3-df65-43c2-a847-698064ae402d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:19.347204 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b019e3-df65-43c2-a847-698064ae402d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:19.348076 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b019e3-df65-43c2-a847-698064ae402d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:19.350079 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b019e3-df65-43c2-a847-698064ae402d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:19.364447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-56b019e3-df65-43c2-a847-698064ae402d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 20:24:19.364447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:19.364447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:24:19.364447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:24:19.364447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:24:19.364447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:24:19.364447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:24:19.364447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:24:19.364447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 20:24:19.364447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:24:19.364447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:24:19.364447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:24:19.364447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:19.364447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:19.364447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:19.364447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:19.364447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:19.365517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:19.365517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:19.365517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:19.365517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:19.365517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:19.365517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:19.365517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:24:19.365517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:24:19.365517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:24:19.365517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:24:19.365517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:19.365517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:19.365517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:19.365517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:19.365517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:19.365517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:19.366388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:19.366388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:19.366388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:19.366388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:19.366388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:19.366388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:19.366388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:19.366388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:19.366388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:19.366388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:19.366388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:24:19.366388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:24:19.366388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 20:24:19.366388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:24:19.366388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 637/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 14 20:24:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:19.374305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2618cafb-8928-46fc-8993-605624044b73 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:19.480146 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2618cafb-8928-46fc-8993-605624044b73 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:19.513807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d736da-8ce9-47a8-882f-9d599427ea01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:19.514461 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d736da-8ce9-47a8-882f-9d599427ea01 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:19.514744 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d736da-8ce9-47a8-882f-9d599427ea01 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:19.515046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d736da-8ce9-47a8-882f-9d599427ea01 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:19.515761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d736da-8ce9-47a8-882f-9d599427ea01 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:19.516064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d736da-8ce9-47a8-882f-9d599427ea01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:19.517425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d736da-8ce9-47a8-882f-9d599427ea01 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:19.522445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 654/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Jul 14 20:24:19 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:19.529097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b019e3-df65-43c2-a847-698064ae402d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:19.626022 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b019e3-df65-43c2-a847-698064ae402d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:19.659402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ffec8-cc88-4b61-8fab-c458c0afcb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:19.660055 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ffec8-cc88-4b61-8fab-c458c0afcb56 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:19.660335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ffec8-cc88-4b61-8fab-c458c0afcb56 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:19.660605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ffec8-cc88-4b61-8fab-c458c0afcb56 None admin] PATH_INFO: `/v3/services` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:19.661302 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ffec8-cc88-4b61-8fab-c458c0afcb56 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:19.661599 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ffec8-cc88-4b61-8fab-c458c0afcb56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:19.662978 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ffec8-cc88-4b61-8fab-c458c0afcb56 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:19.667949 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 638/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Jul 14 20:24:19 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:21.672013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2d8f6-af24-4d26-8d34-d86a940f3087 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:21.672495 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2d8f6-af24-4d26-8d34-d86a940f3087 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:21.672605 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2d8f6-af24-4d26-8d34-d86a940f3087 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:21.673919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 655/1293] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:24:21.679299 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d7b9a-9301-43c3-a5c6-493646d773af None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:21.679825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d7b9a-9301-43c3-a5c6-493646d773af None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:21.679825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d7b9a-9301-43c3-a5c6-493646d773af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:21.696981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-601d7b9a-9301-43c3-a5c6-493646d773af None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:24:21.698168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-601d7b9a-9301-43c3-a5c6-493646d773af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:24:21.727771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:21.765385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 639/1294] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:24:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:24:21.787926 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2d8f6-af24-4d26-8d34-d86a940f3087 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:21.820820 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:21.890645 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2d8f6-af24-4d26-8d34-d86a940f3087 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:21.923335 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4718ea1-3c52-4b63-8f98-5fa2492fa7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:21.923995 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4718ea1-3c52-4b63-8f98-5fa2492fa7d3 None admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:21.924278 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4718ea1-3c52-4b63-8f98-5fa2492fa7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:21.924554 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4718ea1-3c52-4b63-8f98-5fa2492fa7d3 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:21.925275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4718ea1-3c52-4b63-8f98-5fa2492fa7d3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:21.926637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4718ea1-3c52-4b63-8f98-5fa2492fa7d3 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:21.939797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 656/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 14 20:24:21 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:24:23.963188 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfbf0b-13c0-471f-8fd5-bd6aeb21ac92 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:23.963695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfbf0b-13c0-471f-8fd5-bd6aeb21ac92 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:23.963695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfbf0b-13c0-471f-8fd5-bd6aeb21ac92 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:23.965026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 640/1296] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:24:23.970154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d625a-c1e8-430a-83be-99df700a75f4 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:23.970520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d625a-c1e8-430a-83be-99df700a75f4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:23.970716 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d625a-c1e8-430a-83be-99df700a75f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:23.987518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-421d625a-c1e8-430a-83be-99df700a75f4 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:24:23.988696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-421d625a-c1e8-430a-83be-99df700a75f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:24:24.060049 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 657/1297] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:24:23 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:24:24.081970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfbf0b-13c0-471f-8fd5-bd6aeb21ac92 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:24.181984 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfbf0b-13c0-471f-8fd5-bd6aeb21ac92 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:24.215284 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466073a7-b68d-47aa-b832-a35be4460a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:24.215995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466073a7-b68d-47aa-b832-a35be4460a64 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:24.216281 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466073a7-b68d-47aa-b832-a35be4460a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:24.216556 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466073a7-b68d-47aa-b832-a35be4460a64 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:24.217202 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466073a7-b68d-47aa-b832-a35be4460a64 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:24.218577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466073a7-b68d-47aa-b832-a35be4460a64 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:24.232459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-466073a7-b68d-47aa-b832-a35be4460a64 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 20:24:24.232459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:24.232459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:24:24.232459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:24:24.232459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:24:24.232459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:24:24.232459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:24:24.232459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:24:24.232459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 20:24:24.232459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:24:24.232459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:24:24.232459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:24:24.232459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:24.232459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:24.232459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:24.232459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:24.232459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:24.233634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:24.233634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:24.233634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:24.233634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:24.233634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:24.233634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:24.233634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:24:24.233634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:24:24.233634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:24:24.233634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:24:24.233634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:24.233634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:24.233634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:24.233634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:24.233634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:24.233634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:24.234653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:24.234653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:24.234653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:24.234653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:24.234653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:24.234653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:24.234653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:24.234653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:24.234653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:24.234653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:24.234653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:24:24.234653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:24:24.234653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 20:24:24.234653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:24:24.234653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 641/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 14 20:24:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:24.239983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d625a-c1e8-430a-83be-99df700a75f4 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:24.258024 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:24.350488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d625a-c1e8-430a-83be-99df700a75f4 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:24.394237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4be71-8473-4459-9068-1fa66ac550c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:24.394907 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4be71-8473-4459-9068-1fa66ac550c1 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:24.395248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4be71-8473-4459-9068-1fa66ac550c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:24.395524 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4be71-8473-4459-9068-1fa66ac550c1 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:24.396242 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4be71-8473-4459-9068-1fa66ac550c1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:24.396551 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4be71-8473-4459-9068-1fa66ac550c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:24.397914 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4be71-8473-4459-9068-1fa66ac550c1 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:24.403670 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 658/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Jul 14 20:24:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:24.410042 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466073a7-b68d-47aa-b832-a35be4460a64 None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:24.440320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:24.507028 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466073a7-b68d-47aa-b832-a35be4460a64 None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:24.539852 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93044311-72f8-416b-ae2c-22c4a7d82349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:24.540491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93044311-72f8-416b-ae2c-22c4a7d82349 None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:24.540773 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93044311-72f8-416b-ae2c-22c4a7d82349 None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:24.541042 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93044311-72f8-416b-ae2c-22c4a7d82349 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:24.541808 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93044311-72f8-416b-ae2c-22c4a7d82349 None admin] RBAC: Adding query filter params (interface=public, service_id=28046df28f4e41a2bc621a7eb81a4960, region_id=RegionOne) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:24.542111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93044311-72f8-416b-ae2c-22c4a7d82349 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:24.543484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93044311-72f8-416b-ae2c-22c4a7d82349 None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:24.548383 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 642/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 14 20:24:24 2022] GET /identity/v3/endpoints?service_id=28046df28f4e41a2bc621a7eb81a4960&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:24.553970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4be71-8473-4459-9068-1fa66ac550c1 None admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:24.637114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:24.655995 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4be71-8473-4459-9068-1fa66ac550c1 None admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:24.688461 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ad94d-bc1b-4b7a-81d3-04f16fd6522f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:24.689115 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ad94d-bc1b-4b7a-81d3-04f16fd6522f None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:24.689396 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ad94d-bc1b-4b7a-81d3-04f16fd6522f None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:24.689667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ad94d-bc1b-4b7a-81d3-04f16fd6522f None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:24.690343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ad94d-bc1b-4b7a-81d3-04f16fd6522f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:24.691733 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ad94d-bc1b-4b7a-81d3-04f16fd6522f None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:24.699666 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 659/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 14 20:24:24 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:24:26.687663 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f04b5-5a7d-431e-b944-ed7c26fd365d None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:26.688214 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f04b5-5a7d-431e-b944-ed7c26fd365d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:26.688214 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f04b5-5a7d-431e-b944-ed7c26fd365d None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:26.689566 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 643/1302] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:24:26.694684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce48a24-79b1-4a87-bb45-9af5250af852 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:26.695178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce48a24-79b1-4a87-bb45-9af5250af852 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:26.695281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce48a24-79b1-4a87-bb45-9af5250af852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:26.712766 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-8ce48a24-79b1-4a87-bb45-9af5250af852 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:24:26.713953 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-8ce48a24-79b1-4a87-bb45-9af5250af852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:24:26.779722 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 660/1303] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:24:26 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 20:24:26.801246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f04b5-5a7d-431e-b944-ed7c26fd365d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:26.894587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:26.902006 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f04b5-5a7d-431e-b944-ed7c26fd365d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:26.934264 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a1443-b2d0-4a10-ba97-5b4103b8aa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:26.935078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a1443-b2d0-4a10-ba97-5b4103b8aa4d None admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:26.935359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a1443-b2d0-4a10-ba97-5b4103b8aa4d None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:26.935631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a1443-b2d0-4a10-ba97-5b4103b8aa4d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:26.936275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a1443-b2d0-4a10-ba97-5b4103b8aa4d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:26.937623 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a1443-b2d0-4a10-ba97-5b4103b8aa4d None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:26.949714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application [None req-8b5a1443-b2d0-4a10-ba97-5b4103b8aa4d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 20:24:26.949714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:26.949714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 20:24:26.949714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 20:24:26.949714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 20:24:26.949714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 20:24:26.949714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 20:24:26.949714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:24:26.949714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 20:24:26.949714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:24:26.949714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 20:24:26.949714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:24:26.949714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 20:24:26.949714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 20:24:26.949714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 20:24:26.949714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 20:24:26.949714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 20:24:26.950904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 20:24:26.950904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 20:24:26.950904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 20:24:26.950904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 20:24:26.950904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 20:24:26.950904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 20:24:26.950904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 20:24:26.950904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 20:24:26.950904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 20:24:26.950904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 20:24:26.950904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 20:24:26.950904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 20:24:26.950904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 20:24:26.950904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 20:24:26.950904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 20:24:26.950904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 20:24:26.951981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 20:24:26.951981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application with Lock( Jul 14 20:24:26.951981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 20:24:26.951981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self._enter() Jul 14 20:24:26.951981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 20:24:26.951981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 20:24:26.951981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 20:24:26.951981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application return self.creator() Jul 14 20:24:26.951981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 20:24:26.951981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application created_value = creator( Jul 14 20:24:26.951981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 20:24:26.951981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 20:24:26.951981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 20:24:26.951981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: ERROR keystone.server.flask.application Jul 14 20:24:26.951981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 644/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 14 20:24:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 20:24:26.957149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce48a24-79b1-4a87-bb45-9af5250af852 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:27.031036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:27.053553 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce48a24-79b1-4a87-bb45-9af5250af852 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:27.087510 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2165c-5601-4011-8e0c-a2f840eaf9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:27.088167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2165c-5601-4011-8e0c-a2f840eaf9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:27.088450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2165c-5601-4011-8e0c-a2f840eaf9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:27.088725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2165c-5601-4011-8e0c-a2f840eaf9c0 None admin] PATH_INFO: `/v3/services` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:27.089429 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2165c-5601-4011-8e0c-a2f840eaf9c0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:24:27.089736 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2165c-5601-4011-8e0c-a2f840eaf9c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:27.091123 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2165c-5601-4011-8e0c-a2f840eaf9c0 None admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:27.096130 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 661/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Jul 14 20:24:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:24:27.101981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a1443-b2d0-4a10-ba97-5b4103b8aa4d None admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:24:27.198156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a1443-b2d0-4a10-ba97-5b4103b8aa4d None admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:24:27.231121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ad5c9-9a38-4c42-b4fa-e1e76f50978a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:24:27.231769 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ad5c9-9a38-4c42-b4fa-e1e76f50978a None admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:24:27.232053 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ad5c9-9a38-4c42-b4fa-e1e76f50978a None admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:24:27.232329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ad5c9-9a38-4c42-b4fa-e1e76f50978a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:24:27.233073 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ad5c9-9a38-4c42-b4fa-e1e76f50978a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:24:27.234434 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ad5c9-9a38-4c42-b4fa-e1e76f50978a None admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:24:27.252384 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 645/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 14 20:24:27 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:24:37.093173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=30 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:24:37.248011 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:30.692665 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8a4f9-8a1b-46ea-9e01-90aa47f04792 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:30.693769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8a4f9-8a1b-46ea-9e01-90aa47f04792 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:30.693769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8a4f9-8a1b-46ea-9e01-90aa47f04792 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:30.695084 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 662/1307] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:25:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:25:30.700648 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f565c-ec80-4492-9b2c-d59a9f5c9b22 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:30.701167 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f565c-ec80-4492-9b2c-d59a9f5c9b22 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:30.701279 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f565c-ec80-4492-9b2c-d59a9f5c9b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:30.788634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-9c1f565c-ec80-4492-9b2c-d59a9f5c9b22 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:25:30.789804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9c1f565c-ec80-4492-9b2c-d59a9f5c9b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:30.923329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 646/1308] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:25:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:25:30.988611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdc89b-43e1-455c-8d7f-fb22748fe46f None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:30.988905 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdc89b-43e1-455c-8d7f-fb22748fe46f None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:30.989187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdc89b-43e1-455c-8d7f-fb22748fe46f None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:30.991327 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 663/1309] 10.222.0.52 () {54 vars in 964 bytes} [Thu Jul 14 20:25:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:25:31.002098 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885106f5-f380-4646-b1b8-b1a1fbbdee29 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:31.002811 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885106f5-f380-4646-b1b8-b1a1fbbdee29 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:31.003553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885106f5-f380-4646-b1b8-b1a1fbbdee29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:31.116510 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-885106f5-f380-4646-b1b8-b1a1fbbdee29 None None] MFA Rules not processed for user `f88257850ee34ad895208ab935b6e643`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:25:31.117860 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-885106f5-f380-4646-b1b8-b1a1fbbdee29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:31.142216 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:31.203977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 647/1310] 10.222.0.52 () {58 vars in 1061 bytes} [Thu Jul 14 20:25:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:25:31.213518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fdc89b-43e1-455c-8d7f-fb22748fe46f None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:25:31.215304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-72fdc89b-43e1-455c-8d7f-fb22748fe46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:31.342808 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fdc89b-43e1-455c-8d7f-fb22748fe46f None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:25:31.425204 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d253d-705c-4c09-9ef5-8992e7c7913c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:25:31.425774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d253d-705c-4c09-9ef5-8992e7c7913c service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:31.426056 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d253d-705c-4c09-9ef5-8992e7c7913c service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:31.426330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d253d-705c-4c09-9ef5-8992e7c7913c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:31.428257 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-0e3d253d-705c-4c09-9ef5-8992e7c7913c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:31.486174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:31.546049 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d253d-705c-4c09-9ef5-8992e7c7913c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:25:31.547523 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d253d-705c-4c09-9ef5-8992e7c7913c service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:25:31.567490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 664/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:25:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:25:33.893098 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970c26d-4c43-4775-b4d4-19a92a0749c8 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:33.893596 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970c26d-4c43-4775-b4d4-19a92a0749c8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:33.893596 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970c26d-4c43-4775-b4d4-19a92a0749c8 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:33.895154 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 648/1312] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:25:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:25:33.900793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe278ff-0117-40e6-982f-90f7a6739c05 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:33.901245 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe278ff-0117-40e6-982f-90f7a6739c05 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:33.901481 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe278ff-0117-40e6-982f-90f7a6739c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:33.989298 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-1fe278ff-0117-40e6-982f-90f7a6739c05 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:25:33.991173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-1fe278ff-0117-40e6-982f-90f7a6739c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:34.103194 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 665/1313] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:25:34.143654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970c26d-4c43-4775-b4d4-19a92a0749c8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:25:34.145156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e970c26d-4c43-4775-b4d4-19a92a0749c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:34.249610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970c26d-4c43-4775-b4d4-19a92a0749c8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:25:34.323196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:34.326457 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9763fbd-ea29-4a00-9c30-77851199d0dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:25:34.327434 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9763fbd-ea29-4a00-9c30-77851199d0dd service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:34.327949 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9763fbd-ea29-4a00-9c30-77851199d0dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:34.328310 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9763fbd-ea29-4a00-9c30-77851199d0dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:34.330359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c9763fbd-ea29-4a00-9c30-77851199d0dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:34.447468 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9763fbd-ea29-4a00-9c30-77851199d0dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:25:34.448915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9763fbd-ea29-4a00-9c30-77851199d0dd service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:25:34.467052 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 649/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:25:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:25:36.809278 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a66d62-a3e1-4ceb-acc7-8526d00e5d23 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:36.809819 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a66d62-a3e1-4ceb-acc7-8526d00e5d23 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:36.809819 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a66d62-a3e1-4ceb-acc7-8526d00e5d23 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:36.811325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 666/1315] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:25:36.817745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bee8d-9e81-4294-8e99-cdfff343b047 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:36.818912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bee8d-9e81-4294-8e99-cdfff343b047 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:36.819063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bee8d-9e81-4294-8e99-cdfff343b047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:36.891262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ec6bee8d-9e81-4294-8e99-cdfff343b047 None None] MFA Rules not processed for user `2f2bd63cb926471a9ba1d25571f4dd51`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:25:36.892660 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ec6bee8d-9e81-4294-8e99-cdfff343b047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:36.951074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:37.004558 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 650/1316] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:25:36 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:25:37.044700 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a66d62-a3e1-4ceb-acc7-8526d00e5d23 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:25:37.046240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-82a66d62-a3e1-4ceb-acc7-8526d00e5d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:37.057462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:37.156129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a66d62-a3e1-4ceb-acc7-8526d00e5d23 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:25:37.229575 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a61291-e800-452a-bcab-bc8fe701b8a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:25:37.230231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a61291-e800-452a-bcab-bc8fe701b8a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:37.230514 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a61291-e800-452a-bcab-bc8fe701b8a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:37.230789 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a61291-e800-452a-bcab-bc8fe701b8a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:37.232478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-86a61291-e800-452a-bcab-bc8fe701b8a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:37.296173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:37.343555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a61291-e800-452a-bcab-bc8fe701b8a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:25:37.345045 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a61291-e800-452a-bcab-bc8fe701b8a6 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:25:37.367685 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 667/1317] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:25:37 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:25:39.943622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d8a68-55be-4ee0-b1aa-e0bd265ee6c7 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:39.944136 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d8a68-55be-4ee0-b1aa-e0bd265ee6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:39.944136 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d8a68-55be-4ee0-b1aa-e0bd265ee6c7 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:39.945483 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 651/1318] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:25:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:25:39.950823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be149639-ff94-4d3d-b7e7-61b9ddd8a867 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:39.951272 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be149639-ff94-4d3d-b7e7-61b9ddd8a867 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:39.951414 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be149639-ff94-4d3d-b7e7-61b9ddd8a867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:40.014513 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-be149639-ff94-4d3d-b7e7-61b9ddd8a867 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:25:40.015750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-be149639-ff94-4d3d-b7e7-61b9ddd8a867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:40.120859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 668/1319] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:25:39 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:25:40.174750 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bfaea-69c3-4903-91da-a1ff4939d030 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:40.175159 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bfaea-69c3-4903-91da-a1ff4939d030 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:40.175447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bfaea-69c3-4903-91da-a1ff4939d030 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:40.176835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 652/1320] 10.222.0.52 () {54 vars in 964 bytes} [Thu Jul 14 20:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:25:40.184789 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cf9ec-bcb1-484f-b7f1-db9acfa1d33c None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:40.185124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cf9ec-bcb1-484f-b7f1-db9acfa1d33c None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:40.185350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cf9ec-bcb1-484f-b7f1-db9acfa1d33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:40.205505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:40.274689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-e38cf9ec-bcb1-484f-b7f1-db9acfa1d33c None None] MFA Rules not processed for user `f88257850ee34ad895208ab935b6e643`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:25:40.276077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e38cf9ec-bcb1-484f-b7f1-db9acfa1d33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:40.354170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 669/1321] 10.222.0.52 () {58 vars in 1061 bytes} [Thu Jul 14 20:25:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:25:40.363303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bfaea-69c3-4903-91da-a1ff4939d030 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:25:40.365486 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-524bfaea-69c3-4903-91da-a1ff4939d030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:40.467786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bfaea-69c3-4903-91da-a1ff4939d030 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:25:40.541601 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746a451-ea2f-46cb-a2cf-6234fee20b53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:25:40.542241 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746a451-ea2f-46cb-a2cf-6234fee20b53 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:40.542527 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746a451-ea2f-46cb-a2cf-6234fee20b53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:40.542797 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746a451-ea2f-46cb-a2cf-6234fee20b53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:40.544487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8746a451-ea2f-46cb-a2cf-6234fee20b53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:40.552004 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:40.653994 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746a451-ea2f-46cb-a2cf-6234fee20b53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:25:40.655490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746a451-ea2f-46cb-a2cf-6234fee20b53 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:25:40.679484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 653/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:25:40 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:25:43.851141 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cce832-2ade-4d93-a652-a6ff038efcfc None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:43.851692 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cce832-2ade-4d93-a652-a6ff038efcfc None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:43.851692 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cce832-2ade-4d93-a652-a6ff038efcfc None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:43.853090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 670/1323] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:25:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:25:43.858823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105195f3-0bae-488b-b5bd-47dbed750160 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:43.859386 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105195f3-0bae-488b-b5bd-47dbed750160 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:43.859489 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105195f3-0bae-488b-b5bd-47dbed750160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:43.932855 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-105195f3-0bae-488b-b5bd-47dbed750160 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:25:43.934012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-105195f3-0bae-488b-b5bd-47dbed750160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:43.998651 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:44.048915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 654/1324] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:25:44.087694 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cce832-2ade-4d93-a652-a6ff038efcfc None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:25:44.089385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e2cce832-2ade-4d93-a652-a6ff038efcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:44.204212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cce832-2ade-4d93-a652-a6ff038efcfc None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:25:44.218446 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:44.284874 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca0585-cb3e-425c-a3b6-57c69b95467d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:25:44.285536 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca0585-cb3e-425c-a3b6-57c69b95467d service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:44.285817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca0585-cb3e-425c-a3b6-57c69b95467d service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:44.286091 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca0585-cb3e-425c-a3b6-57c69b95467d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:44.287868 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-2aca0585-cb3e-425c-a3b6-57c69b95467d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:44.411813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca0585-cb3e-425c-a3b6-57c69b95467d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:25:44.413297 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca0585-cb3e-425c-a3b6-57c69b95467d service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:25:44.432366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 671/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:25:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:25:46.546589 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f063618-1cd1-45f0-b59c-b28dadf70a05 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:46.547178 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f063618-1cd1-45f0-b59c-b28dadf70a05 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:46.547178 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f063618-1cd1-45f0-b59c-b28dadf70a05 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:46.548554 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 655/1326] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:25:46.554438 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32340d1-7432-4362-bb17-c2d4771da0f6 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:46.555091 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32340d1-7432-4362-bb17-c2d4771da0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:46.555091 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32340d1-7432-4362-bb17-c2d4771da0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:46.601608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:46.624747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-b32340d1-7432-4362-bb17-c2d4771da0f6 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:25:46.625887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b32340d1-7432-4362-bb17-c2d4771da0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:46.740820 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 672/1327] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:25:46.778148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f063618-1cd1-45f0-b59c-b28dadf70a05 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:25:46.779719 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6f063618-1cd1-45f0-b59c-b28dadf70a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:46.887406 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f063618-1cd1-45f0-b59c-b28dadf70a05 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:25:46.963891 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145139b-c09a-4188-b645-149c0fa45f65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:25:46.964546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145139b-c09a-4188-b645-149c0fa45f65 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:46.964839 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145139b-c09a-4188-b645-149c0fa45f65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:46.965117 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145139b-c09a-4188-b645-149c0fa45f65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:46.966755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7145139b-c09a-4188-b645-149c0fa45f65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:46.984080 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:47.084394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145139b-c09a-4188-b645-149c0fa45f65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:25:47.085855 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145139b-c09a-4188-b645-149c0fa45f65 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:25:47.106004 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 656/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:25:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:25:49.288167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11b9cd-8865-4009-9654-4f14f7564a37 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:49.288952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11b9cd-8865-4009-9654-4f14f7564a37 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:49.289142 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11b9cd-8865-4009-9654-4f14f7564a37 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:49.291789 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 673/1329] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:25:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:25:49.299721 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ecc19-aa39-487c-8951-4921c7bb858a None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:49.300511 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ecc19-aa39-487c-8951-4921c7bb858a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:49.300792 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ecc19-aa39-487c-8951-4921c7bb858a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:49.377656 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-246ecc19-aa39-487c-8951-4921c7bb858a None None] MFA Rules not processed for user `2f2bd63cb926471a9ba1d25571f4dd51`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:25:49.379076 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-246ecc19-aa39-487c-8951-4921c7bb858a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:49.455239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:49.510833 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 657/1330] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:25:49 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:25:49.552494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11b9cd-8865-4009-9654-4f14f7564a37 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:25:49.555317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6f11b9cd-8865-4009-9654-4f14f7564a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:49.711145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11b9cd-8865-4009-9654-4f14f7564a37 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:25:49.736165 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:49.822211 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb33eb0-daed-43ca-86b5-e0560c4d0f4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:25:49.823393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb33eb0-daed-43ca-86b5-e0560c4d0f4a service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:49.823924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb33eb0-daed-43ca-86b5-e0560c4d0f4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:49.824426 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb33eb0-daed-43ca-86b5-e0560c4d0f4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:49.827621 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6eb33eb0-daed-43ca-86b5-e0560c4d0f4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:49.998030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb33eb0-daed-43ca-86b5-e0560c4d0f4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:25:50.000775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb33eb0-daed-43ca-86b5-e0560c4d0f4a service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:25:50.035548 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 674/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:25:49 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:25:52.470648 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78fc25-1675-4549-abc4-aab08e80e8d2 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:52.471238 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78fc25-1675-4549-abc4-aab08e80e8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:52.471238 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78fc25-1675-4549-abc4-aab08e80e8d2 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:52.472630 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 658/1332] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:25:52.478739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a1edb-b82a-4ebd-8a50-945cbc2d795b None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:52.479282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a1edb-b82a-4ebd-8a50-945cbc2d795b None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:52.479375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a1edb-b82a-4ebd-8a50-945cbc2d795b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:52.549735 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:52.553100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-f41a1edb-b82a-4ebd-8a50-945cbc2d795b None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:25:52.554969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f41a1edb-b82a-4ebd-8a50-945cbc2d795b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:52.695407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 675/1333] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:25:52.733033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78fc25-1675-4549-abc4-aab08e80e8d2 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:25:52.735647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3a78fc25-1675-4549-abc4-aab08e80e8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:52.884367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78fc25-1675-4549-abc4-aab08e80e8d2 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:25:52.963945 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe4bf6-1ea5-4069-9379-325a0f8a559a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:25:52.964513 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe4bf6-1ea5-4069-9379-325a0f8a559a service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:52.964791 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe4bf6-1ea5-4069-9379-325a0f8a559a service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:52.965063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe4bf6-1ea5-4069-9379-325a0f8a559a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:52.966723 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-50fe4bf6-1ea5-4069-9379-325a0f8a559a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:52.996068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:53.102215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe4bf6-1ea5-4069-9379-325a0f8a559a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:25:53.103700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe4bf6-1ea5-4069-9379-325a0f8a559a service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:25:53.123884 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 659/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:25:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:25:55.332452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b4cf7-69e1-4df2-8ad9-867f4e990412 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:55.333062 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b4cf7-69e1-4df2-8ad9-867f4e990412 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:55.333062 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b4cf7-69e1-4df2-8ad9-867f4e990412 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:55.334474 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 676/1335] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:25:55.341049 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96377064-fe84-461c-b77c-2b530e155930 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:55.341631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96377064-fe84-461c-b77c-2b530e155930 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:55.341631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96377064-fe84-461c-b77c-2b530e155930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:55.418874 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-96377064-fe84-461c-b77c-2b530e155930 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:25:55.420166 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-96377064-fe84-461c-b77c-2b530e155930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:55.511670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:55.540096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 660/1336] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:25:55.586913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b4cf7-69e1-4df2-8ad9-867f4e990412 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:25:55.589723 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-2d3b4cf7-69e1-4df2-8ad9-867f4e990412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:55.731093 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b4cf7-69e1-4df2-8ad9-867f4e990412 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:25:55.771440 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:55.825898 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4dbae-d3b7-4611-8a50-236daf1297d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:25:55.826558 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4dbae-d3b7-4611-8a50-236daf1297d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:55.826844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4dbae-d3b7-4611-8a50-236daf1297d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:55.827160 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4dbae-d3b7-4611-8a50-236daf1297d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:55.828913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f4e4dbae-d3b7-4611-8a50-236daf1297d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:55.959887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4dbae-d3b7-4611-8a50-236daf1297d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:25:55.961279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4dbae-d3b7-4611-8a50-236daf1297d8 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:25:55.983485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 677/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:25:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:25:58.626692 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714049f0-a0c0-4a93-8da5-7cb08d0739b9 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:58.627384 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714049f0-a0c0-4a93-8da5-7cb08d0739b9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:58.627384 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714049f0-a0c0-4a93-8da5-7cb08d0739b9 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:58.628633 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 661/1338] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:25:58.637569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa4947-f3eb-47c1-a695-8083cd70b70f None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:58.638288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa4947-f3eb-47c1-a695-8083cd70b70f None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:58.638288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa4947-f3eb-47c1-a695-8083cd70b70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:58.727889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-d1fa4947-f3eb-47c1-a695-8083cd70b70f None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:25:58.729239 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d1fa4947-f3eb-47c1-a695-8083cd70b70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:58.740237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:58.912976 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 678/1339] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:25:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:25:58.951182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714049f0-a0c0-4a93-8da5-7cb08d0739b9 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:25:58.952767 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-714049f0-a0c0-4a93-8da5-7cb08d0739b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:59.072915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714049f0-a0c0-4a93-8da5-7cb08d0739b9 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:25:59.159732 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6390616-2f79-4cbd-aed7-ebdc3f1af68e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:25:59.160393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6390616-2f79-4cbd-aed7-ebdc3f1af68e service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:25:59.160679 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6390616-2f79-4cbd-aed7-ebdc3f1af68e service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:25:59.161494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6390616-2f79-4cbd-aed7-ebdc3f1af68e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:25:59.163357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c6390616-2f79-4cbd-aed7-ebdc3f1af68e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:25:59.208847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:25:59.297469 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6390616-2f79-4cbd-aed7-ebdc3f1af68e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:25:59.299069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6390616-2f79-4cbd-aed7-ebdc3f1af68e service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:25:59.321667 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 662/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:25:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:26:04.444289 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0424a-58c0-40d0-88ea-d3fa3219726f None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:04.444828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0424a-58c0-40d0-88ea-d3fa3219726f None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:04.445259 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0424a-58c0-40d0-88ea-d3fa3219726f None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:04.448007 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 679/1341] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:26:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:26:04.456089 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c282ce-b77c-4098-a012-4bdf7cb3af68 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:04.456515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c282ce-b77c-4098-a012-4bdf7cb3af68 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:04.456685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c282ce-b77c-4098-a012-4bdf7cb3af68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:04.533212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-49c282ce-b77c-4098-a012-4bdf7cb3af68 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:26:04.534459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-49c282ce-b77c-4098-a012-4bdf7cb3af68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:04.653062 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:04.677098 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 663/1342] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:26:04.736704 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0424a-58c0-40d0-88ea-d3fa3219726f None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:26:04.739354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-75a0424a-58c0-40d0-88ea-d3fa3219726f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:04.904296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0424a-58c0-40d0-88ea-d3fa3219726f None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:26:04.964531 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:05.017576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d9387-b08c-42f3-9db5-a7e776767f6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:26:05.018682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d9387-b08c-42f3-9db5-a7e776767f6c service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:05.019226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d9387-b08c-42f3-9db5-a7e776767f6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:05.019722 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d9387-b08c-42f3-9db5-a7e776767f6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:05.023113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-388d9387-b08c-42f3-9db5-a7e776767f6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:05.181774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d9387-b08c-42f3-9db5-a7e776767f6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:26:05.183518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d9387-b08c-42f3-9db5-a7e776767f6c service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:26:05.216172 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 680/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:26:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:26:08.352859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914855c-2fda-4d50-9b50-c5759bf9af72 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:08.353690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914855c-2fda-4d50-9b50-c5759bf9af72 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:08.353894 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914855c-2fda-4d50-9b50-c5759bf9af72 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:08.356363 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 664/1344] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:26:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:26:08.369364 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e982f58-2612-45fa-bdd9-db1a61f5f75c None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:08.370305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e982f58-2612-45fa-bdd9-db1a61f5f75c None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:08.371375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e982f58-2612-45fa-bdd9-db1a61f5f75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:08.472794 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-7e982f58-2612-45fa-bdd9-db1a61f5f75c None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:26:08.474687 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-7e982f58-2612-45fa-bdd9-db1a61f5f75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:08.509806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:08.658523 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 681/1345] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:26:08 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:26:08.700442 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914855c-2fda-4d50-9b50-c5759bf9af72 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:26:08.702295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e914855c-2fda-4d50-9b50-c5759bf9af72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:08.848115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914855c-2fda-4d50-9b50-c5759bf9af72 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:26:08.980854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65420ca-adc8-4ba8-a247-7e744b664d3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:26:08.981435 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65420ca-adc8-4ba8-a247-7e744b664d3b service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:08.981721 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65420ca-adc8-4ba8-a247-7e744b664d3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:08.981992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65420ca-adc8-4ba8-a247-7e744b664d3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:08.983767 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a65420ca-adc8-4ba8-a247-7e744b664d3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:09.038161 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:09.118315 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65420ca-adc8-4ba8-a247-7e744b664d3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:26:09.119827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65420ca-adc8-4ba8-a247-7e744b664d3b service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:26:09.161822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 665/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:26:08 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:26:14.602604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37add2b0-b02d-4741-b81b-8f607479eef7 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:14.603379 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37add2b0-b02d-4741-b81b-8f607479eef7 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:14.603379 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37add2b0-b02d-4741-b81b-8f607479eef7 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:14.604669 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 682/1347] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:26:14.615258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445bcdb-3189-46b3-b949-7e20057615da None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:14.615916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445bcdb-3189-46b3-b949-7e20057615da None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:14.616408 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445bcdb-3189-46b3-b949-7e20057615da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:14.722256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-2445bcdb-3189-46b3-b949-7e20057615da None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:26:14.724091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2445bcdb-3189-46b3-b949-7e20057615da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:14.850716 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:14.854997 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 666/1348] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:26:14.897491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37add2b0-b02d-4741-b81b-8f607479eef7 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:26:14.899295 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-37add2b0-b02d-4741-b81b-8f607479eef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:15.045213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37add2b0-b02d-4741-b81b-8f607479eef7 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:26:15.125529 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:15.159217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a64a14-737e-4728-be61-4250d5a9998f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:26:15.159887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a64a14-737e-4728-be61-4250d5a9998f service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:15.160734 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a64a14-737e-4728-be61-4250d5a9998f service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:15.161012 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a64a14-737e-4728-be61-4250d5a9998f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:15.162887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d3a64a14-737e-4728-be61-4250d5a9998f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:15.326302 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a64a14-737e-4728-be61-4250d5a9998f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:26:15.328151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a64a14-737e-4728-be61-4250d5a9998f service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:26:15.352185 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 683/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:26:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:26:21.430153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffd3a7-12df-4132-807d-e4d023f8b674 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:21.431755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffd3a7-12df-4132-807d-e4d023f8b674 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:21.432496 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffd3a7-12df-4132-807d-e4d023f8b674 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:21.436780 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 667/1350] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 14 20:26:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:26:21.456518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a05a4-5041-4283-a83e-126214e586c4 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:21.456518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a05a4-5041-4283-a83e-126214e586c4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:21.456518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a05a4-5041-4283-a83e-126214e586c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:21.559427 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-3c3a05a4-5041-4283-a83e-126214e586c4 None None] MFA Rules not processed for user `1afad68819504c2b864851c6989ae25a`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:26:21.560952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3c3a05a4-5041-4283-a83e-126214e586c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:21.602161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:21.834077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 684/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 14 20:26:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:26:28.172737 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900c3f4-b3da-4fd4-9f07-48c3c79659cd None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:28.173329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900c3f4-b3da-4fd4-9f07-48c3c79659cd None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:28.173871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900c3f4-b3da-4fd4-9f07-48c3c79659cd None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:28.177834 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 668/1352] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 14 20:26:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:26:28.185960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698427c-40f7-46ab-beaa-01424c8a585c None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:28.186705 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698427c-40f7-46ab-beaa-01424c8a585c None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:28.187289 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698427c-40f7-46ab-beaa-01424c8a585c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:28.334009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-1698427c-40f7-46ab-beaa-01424c8a585c None None] MFA Rules not processed for user `1afad68819504c2b864851c6989ae25a`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:26:28.335631 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-1698427c-40f7-46ab-beaa-01424c8a585c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:28.447694 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:28.504348 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 685/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 14 20:26:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:26:30.319625 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4c427-0979-4eef-a403-d6f1a86ac6c1 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:30.320408 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4c427-0979-4eef-a403-d6f1a86ac6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:30.320408 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4c427-0979-4eef-a403-d6f1a86ac6c1 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:30.322016 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 669/1354] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 14 20:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:26:30.330887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c51fb-c303-4b55-b374-1e2737f1cff1 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:30.332268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c51fb-c303-4b55-b374-1e2737f1cff1 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:30.332517 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c51fb-c303-4b55-b374-1e2737f1cff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:30.485952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-456c51fb-c303-4b55-b374-1e2737f1cff1 None None] MFA Rules not processed for user `1afad68819504c2b864851c6989ae25a`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:26:30.487245 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-456c51fb-c303-4b55-b374-1e2737f1cff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:30.665840 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 686/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 14 20:26:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:26:33.699893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f312a0-7393-46df-855b-22e04aa8fcbf None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:33.700813 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f312a0-7393-46df-855b-22e04aa8fcbf None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:33.700813 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f312a0-7393-46df-855b-22e04aa8fcbf None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:33.702102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 670/1356] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:26:33.708280 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c31314-7bfc-4c76-99b8-aa7c74ed7b31 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:33.709009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c31314-7bfc-4c76-99b8-aa7c74ed7b31 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:33.709009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c31314-7bfc-4c76-99b8-aa7c74ed7b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:33.774977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:33.830792 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-e8c31314-7bfc-4c76-99b8-aa7c74ed7b31 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:26:33.833023 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e8c31314-7bfc-4c76-99b8-aa7c74ed7b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:33.978535 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 687/1357] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:26:34.098653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118af9c-ec13-40c4-8d08-2a61389c4a74 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:34.099322 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118af9c-ec13-40c4-8d08-2a61389c4a74 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:34.099905 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118af9c-ec13-40c4-8d08-2a61389c4a74 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:34.102373 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 671/1358] 10.222.0.52 () {54 vars in 961 bytes} [Thu Jul 14 20:26:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:26:34.119113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd70d8d-0aca-49be-90e3-324974e24927 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:34.119645 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd70d8d-0aca-49be-90e3-324974e24927 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:34.120148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd70d8d-0aca-49be-90e3-324974e24927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:34.252011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-7dd70d8d-0aca-49be-90e3-324974e24927 None None] MFA Rules not processed for user `aa3bb9f1e24a4e9ebd460e97b3736bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:26:34.253996 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-7dd70d8d-0aca-49be-90e3-324974e24927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:34.317314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:34.412471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 688/1359] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Jul 14 20:26:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:26:34.426011 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118af9c-ec13-40c4-8d08-2a61389c4a74 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:26:34.427882 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f118af9c-ec13-40c4-8d08-2a61389c4a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:34.600840 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118af9c-ec13-40c4-8d08-2a61389c4a74 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:26:34.699417 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:34.761429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdd549-8b29-4bc7-a932-e96f60822e24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:26:34.762489 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdd549-8b29-4bc7-a932-e96f60822e24 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:34.762489 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdd549-8b29-4bc7-a932-e96f60822e24 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:34.762779 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdd549-8b29-4bc7-a932-e96f60822e24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:34.764379 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-85bdd549-8b29-4bc7-a932-e96f60822e24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:34.898301 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bdd549-8b29-4bc7-a932-e96f60822e24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:26:34.902332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bdd549-8b29-4bc7-a932-e96f60822e24 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:26:34.933103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 672/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:26:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:26:37.853829 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576c493-426e-4a50-8ffa-e3b731760173 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:37.854682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576c493-426e-4a50-8ffa-e3b731760173 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:37.854682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576c493-426e-4a50-8ffa-e3b731760173 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:37.856152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 689/1361] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:26:37.861860 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e24f7d-cf38-40b3-8958-076faffad8a7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:37.863141 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e24f7d-cf38-40b3-8958-076faffad8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:37.863141 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e24f7d-cf38-40b3-8958-076faffad8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:37.942865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c9e24f7d-cf38-40b3-8958-076faffad8a7 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:26:37.944565 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c9e24f7d-cf38-40b3-8958-076faffad8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:37.997676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:38.137615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 673/1362] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:26:38.184640 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576c493-426e-4a50-8ffa-e3b731760173 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:26:38.186290 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-c576c493-426e-4a50-8ffa-e3b731760173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:38.353691 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576c493-426e-4a50-8ffa-e3b731760173 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:26:38.364763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:38.510456 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3537c36-202b-4a91-b2dc-8611d44c126d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:26:38.511587 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3537c36-202b-4a91-b2dc-8611d44c126d service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:38.512066 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3537c36-202b-4a91-b2dc-8611d44c126d service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:38.512526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3537c36-202b-4a91-b2dc-8611d44c126d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:38.515830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e3537c36-202b-4a91-b2dc-8611d44c126d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:38.704607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3537c36-202b-4a91-b2dc-8611d44c126d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:26:38.706337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3537c36-202b-4a91-b2dc-8611d44c126d service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:26:38.730243 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 690/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:26:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:26:41.617399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a72e9-f60c-4bab-80fd-aca8858c483f None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:41.618693 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a72e9-f60c-4bab-80fd-aca8858c483f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:41.619165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a72e9-f60c-4bab-80fd-aca8858c483f None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:41.620902 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 674/1364] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:26:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:26:41.628850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8329ce-5e56-4bed-8a09-1cb9bf19e3b5 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:41.630073 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8329ce-5e56-4bed-8a09-1cb9bf19e3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:41.630735 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8329ce-5e56-4bed-8a09-1cb9bf19e3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:41.665650 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:41.770637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-ca8329ce-5e56-4bed-8a09-1cb9bf19e3b5 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:26:41.773250 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ca8329ce-5e56-4bed-8a09-1cb9bf19e3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:41.962837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 691/1365] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:26:42.010588 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee3453-77d1-42fa-8341-b6f87fbe1117 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:42.011034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee3453-77d1-42fa-8341-b6f87fbe1117 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:42.011329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee3453-77d1-42fa-8341-b6f87fbe1117 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:42.013138 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 675/1366] 10.222.0.52 () {54 vars in 961 bytes} [Thu Jul 14 20:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:26:42.020363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda1d80-51dd-463f-b8ef-e0ec46d182b2 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:42.021020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda1d80-51dd-463f-b8ef-e0ec46d182b2 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:42.021400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda1d80-51dd-463f-b8ef-e0ec46d182b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:42.114061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-abda1d80-51dd-463f-b8ef-e0ec46d182b2 None None] MFA Rules not processed for user `aa3bb9f1e24a4e9ebd460e97b3736bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:26:42.115597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-abda1d80-51dd-463f-b8ef-e0ec46d182b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:42.160578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:42.292820 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 692/1367] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Jul 14 20:26:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:26:42.302983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee3453-77d1-42fa-8341-b6f87fbe1117 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:26:42.304997 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-92ee3453-77d1-42fa-8341-b6f87fbe1117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:42.496460 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee3453-77d1-42fa-8341-b6f87fbe1117 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:26:42.533499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:42.685566 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa849e5-f59f-41e1-b7de-ade359870113 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:26:42.686655 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa849e5-f59f-41e1-b7de-ade359870113 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:42.687004 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa849e5-f59f-41e1-b7de-ade359870113 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:42.687412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa849e5-f59f-41e1-b7de-ade359870113 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:42.690776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3aa849e5-f59f-41e1-b7de-ade359870113 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:42.875653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa849e5-f59f-41e1-b7de-ade359870113 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:26:42.877802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa849e5-f59f-41e1-b7de-ade359870113 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:26:42.905331 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 676/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:26:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:26:43.836541 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b55dc-09f0-4cdc-9c91-22cd1bdc628f None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:43.837429 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b55dc-09f0-4cdc-9c91-22cd1bdc628f None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:43.837584 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b55dc-09f0-4cdc-9c91-22cd1bdc628f None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:43.843608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 693/1369] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:26:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:26:43.850036 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92df03-829c-473d-aae3-214cbbc9ba37 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:43.850888 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92df03-829c-473d-aae3-214cbbc9ba37 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:43.851269 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92df03-829c-473d-aae3-214cbbc9ba37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:43.914572 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:43.993941 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ec92df03-829c-473d-aae3-214cbbc9ba37 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:26:43.996520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ec92df03-829c-473d-aae3-214cbbc9ba37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:44.184251 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 677/1370] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:26:44.522401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bb8a8-20a5-4e5d-955c-6a7c96b4b609 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:44.522727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bb8a8-20a5-4e5d-955c-6a7c96b4b609 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:44.523055 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bb8a8-20a5-4e5d-955c-6a7c96b4b609 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:44.524566 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 694/1371] 10.222.0.52 () {54 vars in 963 bytes} [Thu Jul 14 20:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:26:44.530237 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c9281-84f6-4cc7-9471-b6da454e19ca None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:44.530622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c9281-84f6-4cc7-9471-b6da454e19ca None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:44.530835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c9281-84f6-4cc7-9471-b6da454e19ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:44.627135 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ed0c9281-84f6-4cc7-9471-b6da454e19ca None None] MFA Rules not processed for user `01b4142ea6f142df9b1314532c961f75`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:26:44.629267 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ed0c9281-84f6-4cc7-9471-b6da454e19ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:44.692397 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:44.734622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 678/1372] 10.222.0.52 () {58 vars in 1060 bytes} [Thu Jul 14 20:26:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:26:44.745854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bb8a8-20a5-4e5d-955c-6a7c96b4b609 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:26:44.747970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f31bb8a8-20a5-4e5d-955c-6a7c96b4b609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:44.881933 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bb8a8-20a5-4e5d-955c-6a7c96b4b609 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:26:44.933011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:44.995330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0016d-aca8-446c-8af8-24a47730d4f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:26:44.996484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0016d-aca8-446c-8af8-24a47730d4f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:44.996996 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0016d-aca8-446c-8af8-24a47730d4f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:44.997503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0016d-aca8-446c-8af8-24a47730d4f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:45.000896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-1ca0016d-aca8-446c-8af8-24a47730d4f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:45.161483 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0016d-aca8-446c-8af8-24a47730d4f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:26:45.164446 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0016d-aca8-446c-8af8-24a47730d4f9 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:26:45.198096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 695/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:26:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:26:45.469676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c32ce-587b-475d-b788-fb40f7013b2c None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:45.470487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c32ce-587b-475d-b788-fb40f7013b2c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:45.470651 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c32ce-587b-475d-b788-fb40f7013b2c None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:45.473269 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 679/1374] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:26:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:26:45.482122 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb8db1-956d-4159-a1ed-83d24729c460 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:45.482510 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb8db1-956d-4159-a1ed-83d24729c460 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:45.482960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb8db1-956d-4159-a1ed-83d24729c460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:45.571384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-9bcb8db1-956d-4159-a1ed-83d24729c460 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:26:45.572645 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9bcb8db1-956d-4159-a1ed-83d24729c460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:45.602551 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:45.727425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 696/1375] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:26:45.785590 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c32ce-587b-475d-b788-fb40f7013b2c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:26:45.787206 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-525c32ce-587b-475d-b788-fb40f7013b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:46.039828 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c32ce-587b-475d-b788-fb40f7013b2c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:26:46.144268 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:46.275823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafba0a-6fd7-41e7-8b32-6b2eb044545f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:26:46.276677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafba0a-6fd7-41e7-8b32-6b2eb044545f service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:46.277114 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafba0a-6fd7-41e7-8b32-6b2eb044545f service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:46.277507 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafba0a-6fd7-41e7-8b32-6b2eb044545f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:46.279918 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4dafba0a-6fd7-41e7-8b32-6b2eb044545f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:46.415998 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafba0a-6fd7-41e7-8b32-6b2eb044545f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:26:46.418080 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafba0a-6fd7-41e7-8b32-6b2eb044545f service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:26:46.442301 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 680/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:26:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:26:48.795411 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddb48c-4dc8-4de6-b64d-3968c10813dd None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:48.796012 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddb48c-4dc8-4de6-b64d-3968c10813dd None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:48.796012 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddb48c-4dc8-4de6-b64d-3968c10813dd None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:48.797378 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 697/1377] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:26:48.804391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038a7d3-f4f1-40b6-9a3f-a408bb941492 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:48.805714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038a7d3-f4f1-40b6-9a3f-a408bb941492 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:48.806429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038a7d3-f4f1-40b6-9a3f-a408bb941492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:48.953258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8038a7d3-f4f1-40b6-9a3f-a408bb941492 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:26:48.955494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8038a7d3-f4f1-40b6-9a3f-a408bb941492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:48.966465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:49.098169 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 681/1378] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:26:49.178847 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddb48c-4dc8-4de6-b64d-3968c10813dd None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:26:49.180798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-7cddb48c-4dc8-4de6-b64d-3968c10813dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:49.415292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddb48c-4dc8-4de6-b64d-3968c10813dd None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:26:49.422420 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:49.650490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6742b4-52f3-4493-8f8d-4a9ea185c3ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:26:49.651505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6742b4-52f3-4493-8f8d-4a9ea185c3ac service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:49.651975 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6742b4-52f3-4493-8f8d-4a9ea185c3ac service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:49.652141 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6742b4-52f3-4493-8f8d-4a9ea185c3ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:49.653932 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-0b6742b4-52f3-4493-8f8d-4a9ea185c3ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:49.785335 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6742b4-52f3-4493-8f8d-4a9ea185c3ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:26:49.787006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6742b4-52f3-4493-8f8d-4a9ea185c3ac service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:26:49.810402 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 698/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:26:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:26:52.238441 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf3ff0-a7cf-4b55-a943-cd62dcb6c7a8 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:52.239307 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf3ff0-a7cf-4b55-a943-cd62dcb6c7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:52.239307 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf3ff0-a7cf-4b55-a943-cd62dcb6c7a8 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:52.240622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 682/1380] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:26:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:26:52.247552 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca9ccb-cbb1-4cb8-8bf1-9b09b21b65ef None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:52.248192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca9ccb-cbb1-4cb8-8bf1-9b09b21b65ef None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:52.248830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca9ccb-cbb1-4cb8-8bf1-9b09b21b65ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:52.268413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:52.350404 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-3eca9ccb-cbb1-4cb8-8bf1-9b09b21b65ef None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:26:52.352026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3eca9ccb-cbb1-4cb8-8bf1-9b09b21b65ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:52.492128 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 699/1381] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:26:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:26:52.538848 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf3ff0-a7cf-4b55-a943-cd62dcb6c7a8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:26:52.540409 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-47bf3ff0-a7cf-4b55-a943-cd62dcb6c7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:52.720764 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:52.746846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf3ff0-a7cf-4b55-a943-cd62dcb6c7a8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:26:52.931822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684516cd-7f5c-4dc1-8b9b-0b7c7aff06c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:26:52.932640 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684516cd-7f5c-4dc1-8b9b-0b7c7aff06c3 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:52.933031 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684516cd-7f5c-4dc1-8b9b-0b7c7aff06c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:52.933409 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684516cd-7f5c-4dc1-8b9b-0b7c7aff06c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:52.935289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-684516cd-7f5c-4dc1-8b9b-0b7c7aff06c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:53.051531 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684516cd-7f5c-4dc1-8b9b-0b7c7aff06c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:26:53.053330 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684516cd-7f5c-4dc1-8b9b-0b7c7aff06c3 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:26:53.067246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:53.079111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 683/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:26:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:26:56.298924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e97f76-c61b-4012-a12c-da3fe4e98651 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:56.299539 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e97f76-c61b-4012-a12c-da3fe4e98651 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:56.299982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e97f76-c61b-4012-a12c-da3fe4e98651 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:56.302405 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 700/1383] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:26:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:26:56.310691 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965486ce-a580-412c-8a34-488308b087e7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:56.311883 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965486ce-a580-412c-8a34-488308b087e7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:56.311883 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965486ce-a580-412c-8a34-488308b087e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:56.422456 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-965486ce-a580-412c-8a34-488308b087e7 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:26:56.424205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-965486ce-a580-412c-8a34-488308b087e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:56.577761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 684/1384] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:26:56.634885 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e97f76-c61b-4012-a12c-da3fe4e98651 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:26:56.642165 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-87e97f76-c61b-4012-a12c-da3fe4e98651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:56.791981 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:56.891997 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e97f76-c61b-4012-a12c-da3fe4e98651 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:26:57.102555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75f4ab-02c8-4abd-a1ff-df3a65b63cb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:26:57.103270 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75f4ab-02c8-4abd-a1ff-df3a65b63cb7 service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:57.103554 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75f4ab-02c8-4abd-a1ff-df3a65b63cb7 service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:57.103834 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75f4ab-02c8-4abd-a1ff-df3a65b63cb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:57.105563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-2e75f4ab-02c8-4abd-a1ff-df3a65b63cb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:57.169578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:26:57.244388 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e75f4ab-02c8-4abd-a1ff-df3a65b63cb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:26:57.246862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e75f4ab-02c8-4abd-a1ff-df3a65b63cb7 service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:26:57.275456 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 701/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:26:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:26:59.585575 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f9985-f3de-4e93-ac39-6c32f4bdfd83 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:59.586430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f9985-f3de-4e93-ac39-6c32f4bdfd83 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:59.586430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f9985-f3de-4e93-ac39-6c32f4bdfd83 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:59.587914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 685/1386] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:26:59.594043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de2d38-9431-4e64-b46c-c5af55f8245f None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:26:59.594788 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de2d38-9431-4e64-b46c-c5af55f8245f None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:26:59.594788 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de2d38-9431-4e64-b46c-c5af55f8245f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:26:59.697920 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-66de2d38-9431-4e64-b46c-c5af55f8245f None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:26:59.699264 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-66de2d38-9431-4e64-b46c-c5af55f8245f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:26:59.881982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 702/1387] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:26:59.944158 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f9985-f3de-4e93-ac39-6c32f4bdfd83 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:26:59.946638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-612f9985-f3de-4e93-ac39-6c32f4bdfd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:00.081201 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:00.177765 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f9985-f3de-4e93-ac39-6c32f4bdfd83 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:00.323422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00e2c9-9337-4f4a-9968-c8be10163c2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:00.324104 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00e2c9-9337-4f4a-9968-c8be10163c2e service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:00.324471 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00e2c9-9337-4f4a-9968-c8be10163c2e service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:00.324699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00e2c9-9337-4f4a-9968-c8be10163c2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:00.326526 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7b00e2c9-9337-4f4a-9968-c8be10163c2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:00.390955 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:00.460317 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00e2c9-9337-4f4a-9968-c8be10163c2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:00.462405 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00e2c9-9337-4f4a-9968-c8be10163c2e service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:00.495734 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 686/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:26:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:27:02.136029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec4a06-a902-4993-893c-8eb88422b8c7 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:02.136705 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec4a06-a902-4993-893c-8eb88422b8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:02.136866 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec4a06-a902-4993-893c-8eb88422b8c7 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:02.139163 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 703/1389] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:27:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:27:02.148140 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeaac52-1dc6-482b-9f74-e847f71c52ce None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:02.148959 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeaac52-1dc6-482b-9f74-e847f71c52ce None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:02.149115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeaac52-1dc6-482b-9f74-e847f71c52ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:02.275701 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-3eeaac52-1dc6-482b-9f74-e847f71c52ce None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:02.277832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3eeaac52-1dc6-482b-9f74-e847f71c52ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:02.463092 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 687/1390] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:27:02.531191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762cf02-8f23-4e57-bae6-764532ec7a7b None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:02.531483 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762cf02-8f23-4e57-bae6-764532ec7a7b None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:02.531763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762cf02-8f23-4e57-bae6-764532ec7a7b None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:02.533196 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 704/1391] 10.222.0.52 () {54 vars in 963 bytes} [Thu Jul 14 20:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:27:02.539808 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc963fcf-48fb-493d-b27e-729e5bf04c9e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:02.540178 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc963fcf-48fb-493d-b27e-729e5bf04c9e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:02.540509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc963fcf-48fb-493d-b27e-729e5bf04c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:02.559375 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:02.636941 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-dc963fcf-48fb-493d-b27e-729e5bf04c9e None None] MFA Rules not processed for user `4b7a38b4a7754037a00d11dafc88645b`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:02.639054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dc963fcf-48fb-493d-b27e-729e5bf04c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:02.729553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 688/1392] 10.222.0.52 () {58 vars in 1060 bytes} [Thu Jul 14 20:27:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:27:02.738230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762cf02-8f23-4e57-bae6-764532ec7a7b None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:27:02.739913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4762cf02-8f23-4e57-bae6-764532ec7a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:02.780413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:02.871162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762cf02-8f23-4e57-bae6-764532ec7a7b None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:02.888398 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa5be3-6db6-4875-a024-ffbe84e25642 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:02.888863 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa5be3-6db6-4875-a024-ffbe84e25642 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:02.889196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa5be3-6db6-4875-a024-ffbe84e25642 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:02.891234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 689/1393] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:27:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:27:02.898455 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7355b-9d91-4250-99fc-def23816dcdc None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:02.898813 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7355b-9d91-4250-99fc-def23816dcdc None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:02.899226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7355b-9d91-4250-99fc-def23816dcdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:02.969101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c700a-6249-4ca0-b48e-9602c6789d4f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b7a38b4a7754037a00d11dafc88645b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:02.969764 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c700a-6249-4ca0-b48e-9602c6789d4f service glance] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:02.970043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c700a-6249-4ca0-b48e-9602c6789d4f service glance] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:02.970315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c700a-6249-4ca0-b48e-9602c6789d4f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:02.972683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e53c700a-6249-4ca0-b48e-9602c6789d4f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:02.990438 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-6ac7355b-9d91-4250-99fc-def23816dcdc None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:02.991889 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6ac7355b-9d91-4250-99fc-def23816dcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:03.059725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:03.074994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:03.112071 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 690/1394] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:27:03.113855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c700a-6249-4ca0-b48e-9602c6789d4f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:03.115364 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c700a-6249-4ca0-b48e-9602c6789d4f service glance] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:03.136384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 705/1395] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:27:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:27:03.158377 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7355b-9d91-4250-99fc-def23816dcdc None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:27:03.160304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6ac7355b-9d91-4250-99fc-def23816dcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:03.293172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7355b-9d91-4250-99fc-def23816dcdc None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:03.376833 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:03.440448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9b092-4768-4f39-893b-10c2233ef7ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:03.440448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9b092-4768-4f39-893b-10c2233ef7ed service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:03.440448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9b092-4768-4f39-893b-10c2233ef7ed service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:03.440448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9b092-4768-4f39-893b-10c2233ef7ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:03.442147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-32f9b092-4768-4f39-893b-10c2233ef7ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:03.550279 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9b092-4768-4f39-893b-10c2233ef7ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:03.551886 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9b092-4768-4f39-893b-10c2233ef7ed service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:03.575277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 691/1396] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:27:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:27:03.993884 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f911629-c8be-4194-abed-afaeb6e62b00 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:03.994515 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f911629-c8be-4194-abed-afaeb6e62b00 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:03.994802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f911629-c8be-4194-abed-afaeb6e62b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:04.074663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-4f911629-c8be-4194-abed-afaeb6e62b00 None None] MFA Rules not processed for user `4f75b8b484384554ad59aa20535875e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:04.075854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4f911629-c8be-4194-abed-afaeb6e62b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:04.192692 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 706/1397] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 14 20:27:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:27:04.245575 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68a040-6708-4ea1-88e3-5f7a20361676 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:04.246211 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68a040-6708-4ea1-88e3-5f7a20361676 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:04.246211 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68a040-6708-4ea1-88e3-5f7a20361676 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:04.247578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 692/1398] 10.222.0.52 () {54 vars in 942 bytes} [Thu Jul 14 20:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:27:04.256199 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf21725-be32-49ea-99cf-cfe0351e9c71 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:04.256568 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf21725-be32-49ea-99cf-cfe0351e9c71 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:04.257002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf21725-be32-49ea-99cf-cfe0351e9c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:04.324906 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:04.350310 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-abf21725-be32-49ea-99cf-cfe0351e9c71 None None] MFA Rules not processed for user `acc23233929144a5b7b3a1733da19828`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:04.351573 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-abf21725-be32-49ea-99cf-cfe0351e9c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:04.493351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 707/1399] 10.222.0.52 () {58 vars in 1039 bytes} [Thu Jul 14 20:27:04 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:27:04.503489 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68a040-6708-4ea1-88e3-5f7a20361676 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:27:04.505249 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-cf68a040-6708-4ea1-88e3-5f7a20361676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:04.605643 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:04.688563 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68a040-6708-4ea1-88e3-5f7a20361676 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:04.819254 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3286bff-5f93-401a-9b6e-74a1e1210e5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc23233929144a5b7b3a1733da19828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:04.819980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3286bff-5f93-401a-9b6e-74a1e1210e5e service swift] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:04.820201 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3286bff-5f93-401a-9b6e-74a1e1210e5e service swift] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:04.820626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3286bff-5f93-401a-9b6e-74a1e1210e5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:04.822452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a3286bff-5f93-401a-9b6e-74a1e1210e5e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:04.902006 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:04.924395 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3286bff-5f93-401a-9b6e-74a1e1210e5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:04.926646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3286bff-5f93-401a-9b6e-74a1e1210e5e service swift] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:04.936337 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 693/1400] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 14 20:27:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 20:27:05.890823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7bec9-c26e-4256-9efa-f8030bcea4cf None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:05.890823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7bec9-c26e-4256-9efa-f8030bcea4cf None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:05.890823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7bec9-c26e-4256-9efa-f8030bcea4cf None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:05.890823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 708/1401] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:27:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:27:05.895540 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbcf87-bbeb-4597-8516-4cb26655e0d7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:05.896333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbcf87-bbeb-4597-8516-4cb26655e0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:05.896333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbcf87-bbeb-4597-8516-4cb26655e0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:06.005033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-77fbcf87-bbeb-4597-8516-4cb26655e0d7 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:06.011416 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-77fbcf87-bbeb-4597-8516-4cb26655e0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:06.280354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 694/1402] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:27:06.355335 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7bec9-c26e-4256-9efa-f8030bcea4cf None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:27:06.355335 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-cad7bec9-c26e-4256-9efa-f8030bcea4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:06.576704 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:06.658608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7bec9-c26e-4256-9efa-f8030bcea4cf None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:06.879389 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290047f-e9b2-49f8-aff9-72e4fee4cfd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:06.880137 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290047f-e9b2-49f8-aff9-72e4fee4cfd5 service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:06.880440 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290047f-e9b2-49f8-aff9-72e4fee4cfd5 service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:06.880718 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290047f-e9b2-49f8-aff9-72e4fee4cfd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:06.882455 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4290047f-e9b2-49f8-aff9-72e4fee4cfd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:06.993715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:07.022850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290047f-e9b2-49f8-aff9-72e4fee4cfd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:07.024673 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290047f-e9b2-49f8-aff9-72e4fee4cfd5 service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:07.046520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 709/1403] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:27:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 699 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:27:09.411710 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2c14c-df43-436f-a7e8-0db02429ab5e None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:09.413038 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2c14c-df43-436f-a7e8-0db02429ab5e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:09.413539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2c14c-df43-436f-a7e8-0db02429ab5e None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:09.416426 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 695/1404] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:27:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:27:09.424918 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de840c63-5f32-49ae-9ce7-a894552ae846 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:09.425773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de840c63-5f32-49ae-9ce7-a894552ae846 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:09.425773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de840c63-5f32-49ae-9ce7-a894552ae846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:09.515145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-de840c63-5f32-49ae-9ce7-a894552ae846 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:09.516479 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-de840c63-5f32-49ae-9ce7-a894552ae846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:09.681571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 710/1405] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:27:09.730980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2c14c-df43-436f-a7e8-0db02429ab5e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:27:09.732579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5ef2c14c-df43-436f-a7e8-0db02429ab5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:09.822577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:09.954458 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2c14c-df43-436f-a7e8-0db02429ab5e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:10.106206 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49045b-83a9-4cb9-90dc-ec3481d968f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:10.107408 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49045b-83a9-4cb9-90dc-ec3481d968f4 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:10.107930 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49045b-83a9-4cb9-90dc-ec3481d968f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:10.108402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49045b-83a9-4cb9-90dc-ec3481d968f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:10.110146 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8e49045b-83a9-4cb9-90dc-ec3481d968f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:10.139408 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:10.281655 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49045b-83a9-4cb9-90dc-ec3481d968f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:10.283610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49045b-83a9-4cb9-90dc-ec3481d968f4 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:10.315848 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 696/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:27:12.838841 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539bd80-07f1-4df4-995a-d12756baecb1 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:12.839757 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539bd80-07f1-4df4-995a-d12756baecb1 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:12.839757 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539bd80-07f1-4df4-995a-d12756baecb1 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:12.840869 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 711/1407] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:27:12.849555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dc67e-1707-403e-bd6b-e2bf5ec06f90 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:12.850371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dc67e-1707-403e-bd6b-e2bf5ec06f90 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:12.850517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dc67e-1707-403e-bd6b-e2bf5ec06f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:12.943474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-809dc67e-1707-403e-bd6b-e2bf5ec06f90 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:12.944877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-809dc67e-1707-403e-bd6b-e2bf5ec06f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:13.077037 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:13.084530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 697/1408] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:27:13.161860 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539bd80-07f1-4df4-995a-d12756baecb1 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:27:13.164660 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3539bd80-07f1-4df4-995a-d12756baecb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:13.290073 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:13.457705 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539bd80-07f1-4df4-995a-d12756baecb1 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:13.678241 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2e349-0305-41ae-8285-73bfeeb6aaf5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:13.678918 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2e349-0305-41ae-8285-73bfeeb6aaf5 service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:13.679243 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2e349-0305-41ae-8285-73bfeeb6aaf5 service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:13.679528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2e349-0305-41ae-8285-73bfeeb6aaf5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:13.681456 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ecb2e349-0305-41ae-8285-73bfeeb6aaf5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:13.690849 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:13.816040 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2e349-0305-41ae-8285-73bfeeb6aaf5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:13.816040 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2e349-0305-41ae-8285-73bfeeb6aaf5 service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:13.854017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 712/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:27:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:27:23.086570 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:23.848271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=42 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:29.241599 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775b369-413f-44bc-95f1-4e4c949da385 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:29.242265 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775b369-413f-44bc-95f1-4e4c949da385 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:29.242460 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775b369-413f-44bc-95f1-4e4c949da385 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:29.244213 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 698/1410] 10.222.0.52 () {54 vars in 918 bytes} [Thu Jul 14 20:27:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:27:29.250539 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacc1dd-d8f9-4a78-83cc-119e7d1f5682 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:29.251378 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacc1dd-d8f9-4a78-83cc-119e7d1f5682 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:29.251378 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacc1dd-d8f9-4a78-83cc-119e7d1f5682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:29.354972 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-9cacc1dd-d8f9-4a78-83cc-119e7d1f5682 None None] MFA Rules not processed for user `1afad68819504c2b864851c6989ae25a`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:29.356280 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9cacc1dd-d8f9-4a78-83cc-119e7d1f5682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:29.488138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 713/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Thu Jul 14 20:27:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:27:29.505936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d27e69-9051-4e6f-884b-0dbfe9965252 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:29.506308 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d27e69-9051-4e6f-884b-0dbfe9965252 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:29.506666 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d27e69-9051-4e6f-884b-0dbfe9965252 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:29.508207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 699/1412] 10.222.0.52 () {54 vars in 918 bytes} [Thu Jul 14 20:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:27:29.514514 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795d0cb-332b-4dd5-ae4d-252e53a99df4 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:29.514827 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795d0cb-332b-4dd5-ae4d-252e53a99df4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:29.515150 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795d0cb-332b-4dd5-ae4d-252e53a99df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:29.608861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:29.613206 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-b795d0cb-332b-4dd5-ae4d-252e53a99df4 None None] MFA Rules not processed for user `aa3bb9f1e24a4e9ebd460e97b3736bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:29.615257 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b795d0cb-332b-4dd5-ae4d-252e53a99df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:29.759065 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 714/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Thu Jul 14 20:27:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:27:32.434566 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a3232-a58c-438c-9cc4-74a4d3f6ba9e None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:32.435227 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a3232-a58c-438c-9cc4-74a4d3f6ba9e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:32.435375 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a3232-a58c-438c-9cc4-74a4d3f6ba9e None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:32.437114 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 20:27:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:27:32.445010 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344638c-31ba-4280-8000-20dcc4e12395 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:32.445525 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344638c-31ba-4280-8000-20dcc4e12395 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:32.445525 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344638c-31ba-4280-8000-20dcc4e12395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:32.519041 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-a344638c-31ba-4280-8000-20dcc4e12395 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:32.520296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-a344638c-31ba-4280-8000-20dcc4e12395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:32.587442 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:32.637486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 20:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:27:32.684233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a3232-a58c-438c-9cc4-74a4d3f6ba9e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:27:32.686010 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4f8a3232-a58c-438c-9cc4-74a4d3f6ba9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:32.865542 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a3232-a58c-438c-9cc4-74a4d3f6ba9e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:32.948777 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:32.995409 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1073bce-df3c-48ea-beb3-e1a0d07b60cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:32.996079 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1073bce-df3c-48ea-beb3-e1a0d07b60cc service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:32.996361 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1073bce-df3c-48ea-beb3-e1a0d07b60cc service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:32.996635 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1073bce-df3c-48ea-beb3-e1a0d07b60cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:32.998436 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a1073bce-df3c-48ea-beb3-e1a0d07b60cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:33.103022 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1073bce-df3c-48ea-beb3-e1a0d07b60cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:33.104676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1073bce-df3c-48ea-beb3-e1a0d07b60cc service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:33.126607 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 701/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:27:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:27:34.101413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60eb40-04c1-4270-ad89-05d7e29dc0c1 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:34.101989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60eb40-04c1-4270-ad89-05d7e29dc0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:34.101989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60eb40-04c1-4270-ad89-05d7e29dc0c1 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:34.103444 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 20:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:27:34.109528 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78590f0d-bfde-44e9-b0f9-9a6127739b56 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:34.110129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78590f0d-bfde-44e9-b0f9-9a6127739b56 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:34.110129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78590f0d-bfde-44e9-b0f9-9a6127739b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:34.250244 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-78590f0d-bfde-44e9-b0f9-9a6127739b56 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:34.252429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-78590f0d-bfde-44e9-b0f9-9a6127739b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:34.307269 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:34.398343 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 20:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:27:34.555158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60eb40-04c1-4270-ad89-05d7e29dc0c1 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:27:34.556708 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-5a60eb40-04c1-4270-ad89-05d7e29dc0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:34.737068 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60eb40-04c1-4270-ad89-05d7e29dc0c1 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:34.782683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:34.880829 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f44e2-2d74-4df2-a7dc-ede6cde4ead4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:34.881467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f44e2-2d74-4df2-a7dc-ede6cde4ead4 service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:34.881754 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f44e2-2d74-4df2-a7dc-ede6cde4ead4 service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:34.882178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f44e2-2d74-4df2-a7dc-ede6cde4ead4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:34.883912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e07f44e2-2d74-4df2-a7dc-ede6cde4ead4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:34.993230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f44e2-2d74-4df2-a7dc-ede6cde4ead4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:34.994779 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f44e2-2d74-4df2-a7dc-ede6cde4ead4 service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:35.017350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 717/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:27:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:27:35.991614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4becb91-3e3f-434f-a459-f73b92db197e None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:35.992448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4becb91-3e3f-434f-a459-f73b92db197e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:35.992448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4becb91-3e3f-434f-a459-f73b92db197e None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:35.993981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 20:27:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:27:36.000147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef7e26-21b4-4320-acbf-bace3d3f9bf7 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:36.000743 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef7e26-21b4-4320-acbf-bace3d3f9bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:36.000743 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef7e26-21b4-4320-acbf-bace3d3f9bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:36.068563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:36.097376 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-47ef7e26-21b4-4320-acbf-bace3d3f9bf7 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:36.098649 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-47ef7e26-21b4-4320-acbf-bace3d3f9bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:36.230474 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 20:27:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:27:36.406354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4becb91-3e3f-434f-a459-f73b92db197e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:27:36.410739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a4becb91-3e3f-434f-a459-f73b92db197e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:36.707226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4becb91-3e3f-434f-a459-f73b92db197e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:36.723621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:36.848150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063005c-08f8-41ca-a252-480c0bef1e4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:36.848883 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063005c-08f8-41ca-a252-480c0bef1e4e service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:36.849222 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063005c-08f8-41ca-a252-480c0bef1e4e service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:36.849537 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063005c-08f8-41ca-a252-480c0bef1e4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:36.851616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6063005c-08f8-41ca-a252-480c0bef1e4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:36.960185 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063005c-08f8-41ca-a252-480c0bef1e4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:36.961891 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063005c-08f8-41ca-a252-480c0bef1e4e service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:36.985367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 704/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:27:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:27:37.925905 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28902b-5a9a-4b0e-90f4-fc297602aba7 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:37.926517 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28902b-5a9a-4b0e-90f4-fc297602aba7 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:37.926517 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28902b-5a9a-4b0e-90f4-fc297602aba7 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:37.928623 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 20:27:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:27:37.936595 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2169f-c655-4285-ac5f-db9550e590cd None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:37.937445 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2169f-c655-4285-ac5f-db9550e590cd None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:37.937445 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2169f-c655-4285-ac5f-db9550e590cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:37.957743 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:38.026719 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-37e2169f-c655-4285-ac5f-db9550e590cd None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:38.028174 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-37e2169f-c655-4285-ac5f-db9550e590cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:38.155191 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 20:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:27:38.311170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28902b-5a9a-4b0e-90f4-fc297602aba7 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:27:38.312805 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-bd28902b-5a9a-4b0e-90f4-fc297602aba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:38.455020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:38.498603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28902b-5a9a-4b0e-90f4-fc297602aba7 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:38.634855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1dc28-b416-448e-a652-4d165378bbf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:38.635693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1dc28-b416-448e-a652-4d165378bbf2 service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:38.636015 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1dc28-b416-448e-a652-4d165378bbf2 service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:38.636454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1dc28-b416-448e-a652-4d165378bbf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:38.639776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-a0a1dc28-b416-448e-a652-4d165378bbf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:38.731413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:38.748644 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1dc28-b416-448e-a652-4d165378bbf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:38.750291 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1dc28-b416-448e-a652-4d165378bbf2 service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:38.774111 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 720/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:27:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:27:40.228678 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a37a7-02f9-4520-8573-cc622f4c2999 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:40.229277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a37a7-02f9-4520-8573-cc622f4c2999 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:40.229626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a37a7-02f9-4520-8573-cc622f4c2999 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:40.232307 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 20:27:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:27:40.238352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fc300-84b6-4d63-bb3f-09090d50356c None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:40.238979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fc300-84b6-4d63-bb3f-09090d50356c None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:40.239120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fc300-84b6-4d63-bb3f-09090d50356c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:40.329985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-099fc300-84b6-4d63-bb3f-09090d50356c None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:40.332040 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-099fc300-84b6-4d63-bb3f-09090d50356c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:40.481785 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 20:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:27:40.642772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a37a7-02f9-4520-8573-cc622f4c2999 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:27:40.644991 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-427a37a7-02f9-4520-8573-cc622f4c2999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:40.872701 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:40.953476 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a37a7-02f9-4520-8573-cc622f4c2999 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:41.098240 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e4bb1-7a98-45c2-88ae-7ec20aa1e07f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:41.098906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e4bb1-7a98-45c2-88ae-7ec20aa1e07f service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:41.099230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e4bb1-7a98-45c2-88ae-7ec20aa1e07f service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:41.099505 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e4bb1-7a98-45c2-88ae-7ec20aa1e07f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:41.101252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1c5e4bb1-7a98-45c2-88ae-7ec20aa1e07f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:41.163435 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:41.245786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e4bb1-7a98-45c2-88ae-7ec20aa1e07f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:41.248790 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e4bb1-7a98-45c2-88ae-7ec20aa1e07f service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:41.285635 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 707/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:27:42.190735 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30095a28-8eaf-42c9-b492-4de0cbacdf17 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:42.191435 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30095a28-8eaf-42c9-b492-4de0cbacdf17 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:42.191435 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30095a28-8eaf-42c9-b492-4de0cbacdf17 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:42.192832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 20:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:27:42.199895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f21ba-c45c-480b-a7da-97f0e9c6214b None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:42.200379 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f21ba-c45c-480b-a7da-97f0e9c6214b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:42.201186 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f21ba-c45c-480b-a7da-97f0e9c6214b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:42.317100 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-290f21ba-c45c-480b-a7da-97f0e9c6214b None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:42.319127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-290f21ba-c45c-480b-a7da-97f0e9c6214b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:42.458328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 20:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:27:42.621460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30095a28-8eaf-42c9-b492-4de0cbacdf17 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:27:42.623036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-30095a28-8eaf-42c9-b492-4de0cbacdf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:42.691367 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:42.809445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30095a28-8eaf-42c9-b492-4de0cbacdf17 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:42.957441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcce69c-2866-40e5-9124-13224188fb01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:42.958108 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcce69c-2866-40e5-9124-13224188fb01 service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:42.958388 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcce69c-2866-40e5-9124-13224188fb01 service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:42.958659 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcce69c-2866-40e5-9124-13224188fb01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:42.960406 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-bbcce69c-2866-40e5-9124-13224188fb01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:42.968956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:43.088611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcce69c-2866-40e5-9124-13224188fb01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:43.092043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcce69c-2866-40e5-9124-13224188fb01 service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:43.127118 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 723/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:27:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:27:44.072929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14139d-f2e1-465c-8617-bda113ad79db None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:44.073830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14139d-f2e1-465c-8617-bda113ad79db None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:44.073830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14139d-f2e1-465c-8617-bda113ad79db None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:44.075615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 20:27:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:27:44.082497 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df222f-65b8-46d8-8eb0-8a63ebead5de None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:44.083197 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df222f-65b8-46d8-8eb0-8a63ebead5de None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:44.083197 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df222f-65b8-46d8-8eb0-8a63ebead5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:44.176892 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-66df222f-65b8-46d8-8eb0-8a63ebead5de None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:44.181557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-66df222f-65b8-46d8-8eb0-8a63ebead5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:44.272138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:44.309700 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 20:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:27:44.472542 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14139d-f2e1-465c-8617-bda113ad79db None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:27:44.474102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-bf14139d-f2e1-465c-8617-bda113ad79db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:44.494699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:44.712745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14139d-f2e1-465c-8617-bda113ad79db None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:44.817825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:44.853535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66152040-4481-48fc-827e-307e01d26a9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:44.854245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66152040-4481-48fc-827e-307e01d26a9d service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:44.854521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66152040-4481-48fc-827e-307e01d26a9d service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:44.854796 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66152040-4481-48fc-827e-307e01d26a9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:44.856720 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-66152040-4481-48fc-827e-307e01d26a9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:44.966357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66152040-4481-48fc-827e-307e01d26a9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:44.967918 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66152040-4481-48fc-827e-307e01d26a9d service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:44.990198 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 710/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:27:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:27:45.868560 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74364fef-71a0-4cfe-8aa9-5c51b2e2354f None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:45.869169 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74364fef-71a0-4cfe-8aa9-5c51b2e2354f None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:45.869169 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74364fef-71a0-4cfe-8aa9-5c51b2e2354f None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:45.870658 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 20:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:27:45.876587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27c51c-7164-4fbc-89c8-228c63a953d3 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:45.877142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27c51c-7164-4fbc-89c8-228c63a953d3 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:45.877142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27c51c-7164-4fbc-89c8-228c63a953d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:45.969968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-fa27c51c-7164-4fbc-89c8-228c63a953d3 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:45.971305 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fa27c51c-7164-4fbc-89c8-228c63a953d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:46.037986 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:46.112887 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 20:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:27:46.271246 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74364fef-71a0-4cfe-8aa9-5c51b2e2354f None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:27:46.273126 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-74364fef-71a0-4cfe-8aa9-5c51b2e2354f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:46.472830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74364fef-71a0-4cfe-8aa9-5c51b2e2354f None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:46.536699 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:46.616303 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed10b0-6ae3-4b59-8c32-4882a313c405 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:46.616964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed10b0-6ae3-4b59-8c32-4882a313c405 service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:46.617247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed10b0-6ae3-4b59-8c32-4882a313c405 service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:46.617524 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed10b0-6ae3-4b59-8c32-4882a313c405 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:46.619266 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e7ed10b0-6ae3-4b59-8c32-4882a313c405 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:46.724456 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed10b0-6ae3-4b59-8c32-4882a313c405 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:46.726102 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed10b0-6ae3-4b59-8c32-4882a313c405 service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:46.746767 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 726/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:27:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:27:47.709677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36758001-5351-4846-a0f8-41e1ebac4716 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:47.710360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36758001-5351-4846-a0f8-41e1ebac4716 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:47.710360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36758001-5351-4846-a0f8-41e1ebac4716 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:47.711711 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 20:27:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:27:47.718358 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980ae62-1fd0-4abd-98ef-6837faeb660e None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:47.719097 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980ae62-1fd0-4abd-98ef-6837faeb660e None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:47.719097 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980ae62-1fd0-4abd-98ef-6837faeb660e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:47.821558 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-6980ae62-1fd0-4abd-98ef-6837faeb660e None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:47.822865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6980ae62-1fd0-4abd-98ef-6837faeb660e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:47.825549 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:47.956677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 20:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:27:48.117613 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36758001-5351-4846-a0f8-41e1ebac4716 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:27:48.119316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-36758001-5351-4846-a0f8-41e1ebac4716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:48.308161 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36758001-5351-4846-a0f8-41e1ebac4716 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:48.341174 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:48.452910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56593dd3-eab7-4f11-af58-0394f179655b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:48.453348 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56593dd3-eab7-4f11-af58-0394f179655b service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:48.453628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56593dd3-eab7-4f11-af58-0394f179655b service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:48.453970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56593dd3-eab7-4f11-af58-0394f179655b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:48.456518 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-56593dd3-eab7-4f11-af58-0394f179655b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:48.573621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56593dd3-eab7-4f11-af58-0394f179655b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:48.575200 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56593dd3-eab7-4f11-af58-0394f179655b service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:48.597782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 713/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:27:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:27:50.680089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b2965-d8e1-4b23-a223-6971a2227feb None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:50.680741 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b2965-d8e1-4b23-a223-6971a2227feb None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:50.680741 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b2965-d8e1-4b23-a223-6971a2227feb None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:50.682316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 20:27:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:27:50.689192 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb845ea3-cda7-404d-906c-dab9f00953cc None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:50.689802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb845ea3-cda7-404d-906c-dab9f00953cc None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:50.689802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb845ea3-cda7-404d-906c-dab9f00953cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:50.737692 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:50.772383 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-bb845ea3-cda7-404d-906c-dab9f00953cc None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:50.773691 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-bb845ea3-cda7-404d-906c-dab9f00953cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:50.933849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 20:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:27:50.975780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b2965-d8e1-4b23-a223-6971a2227feb None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:27:50.977408 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-235b2965-d8e1-4b23-a223-6971a2227feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:51.100781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b2965-d8e1-4b23-a223-6971a2227feb None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:51.136400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:51.189748 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb54d4-a5d6-42ba-b99b-2f115e0a6919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:51.190981 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb54d4-a5d6-42ba-b99b-2f115e0a6919 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:51.191272 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb54d4-a5d6-42ba-b99b-2f115e0a6919 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:51.191546 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb54d4-a5d6-42ba-b99b-2f115e0a6919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:51.193287 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f3eb54d4-a5d6-42ba-b99b-2f115e0a6919 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:51.367555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb54d4-a5d6-42ba-b99b-2f115e0a6919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:51.370396 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb54d4-a5d6-42ba-b99b-2f115e0a6919 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:51.411550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 729/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:27:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:27:53.905800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a34be4-910e-4097-b1b9-1984e40af8ae None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:53.906351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a34be4-910e-4097-b1b9-1984e40af8ae None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:53.906351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a34be4-910e-4097-b1b9-1984e40af8ae None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:53.907701 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 20:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:27:53.919143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b83eb1-c8b1-423a-a2a4-e25b2cc81973 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:53.919781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b83eb1-c8b1-423a-a2a4-e25b2cc81973 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:53.920009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b83eb1-c8b1-423a-a2a4-e25b2cc81973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:54.012240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-44b83eb1-c8b1-423a-a2a4-e25b2cc81973 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:54.013563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-44b83eb1-c8b1-423a-a2a4-e25b2cc81973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:54.029954 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:54.130395 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 20:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:27:54.172078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a34be4-910e-4097-b1b9-1984e40af8ae None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:27:54.173639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-67a34be4-910e-4097-b1b9-1984e40af8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:54.295613 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a34be4-910e-4097-b1b9-1984e40af8ae None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:54.310169 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:54.382394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf70c7-188f-4dc4-9e98-d4c29cd5e656 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:54.383322 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf70c7-188f-4dc4-9e98-d4c29cd5e656 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:54.383710 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf70c7-188f-4dc4-9e98-d4c29cd5e656 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:54.383897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf70c7-188f-4dc4-9e98-d4c29cd5e656 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:54.385708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-bcdf70c7-188f-4dc4-9e98-d4c29cd5e656 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:54.550263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf70c7-188f-4dc4-9e98-d4c29cd5e656 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:54.552289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf70c7-188f-4dc4-9e98-d4c29cd5e656 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:54.574182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 716/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:27:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:27:57.144913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef502b-5a5d-493e-9a14-091e3ea2cc7c None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:57.145483 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef502b-5a5d-493e-9a14-091e3ea2cc7c None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:57.145483 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef502b-5a5d-493e-9a14-091e3ea2cc7c None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:57.146827 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 20:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:27:57.152261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac5f0e-b70b-4ffb-8ff7-37b16bd6aae3 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:57.152689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac5f0e-b70b-4ffb-8ff7-37b16bd6aae3 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:57.152831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac5f0e-b70b-4ffb-8ff7-37b16bd6aae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:57.199974 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:57.217518 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f7ac5f0e-b70b-4ffb-8ff7-37b16bd6aae3 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:27:57.218738 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f7ac5f0e-b70b-4ffb-8ff7-37b16bd6aae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:57.318246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 20:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:27:57.360112 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef502b-5a5d-493e-9a14-091e3ea2cc7c None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:27:57.361712 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3aef502b-5a5d-493e-9a14-091e3ea2cc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:57.508253 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef502b-5a5d-493e-9a14-091e3ea2cc7c None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:27:57.569961 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:27:57.608763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84d0f0-8d78-47ad-969c-1377a19cbb99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:27:57.609417 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84d0f0-8d78-47ad-969c-1377a19cbb99 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:27:57.609699 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84d0f0-8d78-47ad-969c-1377a19cbb99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:27:57.609983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84d0f0-8d78-47ad-969c-1377a19cbb99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:27:57.612098 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-0b84d0f0-8d78-47ad-969c-1377a19cbb99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:27:57.752546 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84d0f0-8d78-47ad-969c-1377a19cbb99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:27:57.754663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84d0f0-8d78-47ad-969c-1377a19cbb99 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:27:57.778319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 732/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:27:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:28:00.036827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa5187-3221-4b13-b4f0-b5292a9f9153 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:00.037741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa5187-3221-4b13-b4f0-b5292a9f9153 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:00.037741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa5187-3221-4b13-b4f0-b5292a9f9153 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:00.039531 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 20:28:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:28:00.046239 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d055f68-d840-4c76-99f7-8b5f192ca139 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:00.046778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d055f68-d840-4c76-99f7-8b5f192ca139 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:00.046778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d055f68-d840-4c76-99f7-8b5f192ca139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:00.117257 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-3d055f68-d840-4c76-99f7-8b5f192ca139 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:28:00.118491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3d055f68-d840-4c76-99f7-8b5f192ca139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:00.140911 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:00.237167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 20:28:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:28:00.303297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa5187-3221-4b13-b4f0-b5292a9f9153 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:28:00.304884 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0efa5187-3221-4b13-b4f0-b5292a9f9153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:00.426896 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa5187-3221-4b13-b4f0-b5292a9f9153 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:28:00.449914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:00.535605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928b4a2-c1e2-4ffb-a885-43d52f4869c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:28:00.536270 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928b4a2-c1e2-4ffb-a885-43d52f4869c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:00.536551 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928b4a2-c1e2-4ffb-a885-43d52f4869c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:00.536825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928b4a2-c1e2-4ffb-a885-43d52f4869c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:00.538527 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a928b4a2-c1e2-4ffb-a885-43d52f4869c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:00.671052 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928b4a2-c1e2-4ffb-a885-43d52f4869c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:28:00.672622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928b4a2-c1e2-4ffb-a885-43d52f4869c2 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:28:00.696916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 719/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:28:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:28:02.995844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c127ae-6d65-4e30-9961-0ad0f9074c38 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:02.996512 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c127ae-6d65-4e30-9961-0ad0f9074c38 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:02.996512 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c127ae-6d65-4e30-9961-0ad0f9074c38 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:02.997785 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 20:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:28:03.003328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574ca83-13eb-4276-9fd1-0278ea348e46 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:03.003787 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574ca83-13eb-4276-9fd1-0278ea348e46 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:03.003880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574ca83-13eb-4276-9fd1-0278ea348e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:03.073238 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:03.078335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d574ca83-13eb-4276-9fd1-0278ea348e46 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:28:03.080225 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d574ca83-13eb-4276-9fd1-0278ea348e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:03.186908 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 20:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:28:03.225835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c127ae-6d65-4e30-9961-0ad0f9074c38 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:28:03.229366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-00c127ae-6d65-4e30-9961-0ad0f9074c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:03.339649 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c127ae-6d65-4e30-9961-0ad0f9074c38 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:28:03.397476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:03.426687 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f625890-80f4-42ef-8d70-d48e989208d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:28:03.427374 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f625890-80f4-42ef-8d70-d48e989208d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:03.427660 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f625890-80f4-42ef-8d70-d48e989208d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:03.427936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f625890-80f4-42ef-8d70-d48e989208d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:03.429575 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-0f625890-80f4-42ef-8d70-d48e989208d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:03.547735 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f625890-80f4-42ef-8d70-d48e989208d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:28:03.549174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f625890-80f4-42ef-8d70-d48e989208d5 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:28:03.570227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 735/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:28:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:28:05.717356 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac20f2d-c373-4f82-9202-ea8c07c215e9 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:05.717918 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac20f2d-c373-4f82-9202-ea8c07c215e9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:05.717918 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac20f2d-c373-4f82-9202-ea8c07c215e9 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:05.719321 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 20:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:28:05.725126 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe29e07-e3b4-49f3-819a-edd1864dba19 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:05.725576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe29e07-e3b4-49f3-819a-edd1864dba19 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:05.727170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe29e07-e3b4-49f3-819a-edd1864dba19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:05.804259 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-7fe29e07-e3b4-49f3-819a-edd1864dba19 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:28:05.806227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-7fe29e07-e3b4-49f3-819a-edd1864dba19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:05.837708 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:05.911520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 20:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:28:05.952514 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac20f2d-c373-4f82-9202-ea8c07c215e9 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:28:05.954061 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3ac20f2d-c373-4f82-9202-ea8c07c215e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:06.063469 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac20f2d-c373-4f82-9202-ea8c07c215e9 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:28:06.097151 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:06.150452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83c909-cbe3-4211-99a0-60404a526029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:28:06.151127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83c909-cbe3-4211-99a0-60404a526029 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:06.151410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83c909-cbe3-4211-99a0-60404a526029 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:06.151679 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83c909-cbe3-4211-99a0-60404a526029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:06.153295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0f83c909-cbe3-4211-99a0-60404a526029 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:06.266650 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83c909-cbe3-4211-99a0-60404a526029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:28:06.268914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83c909-cbe3-4211-99a0-60404a526029 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:28:06.287630 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 722/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:28:08.391813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ec4c1-1ce8-49e5-bdaa-488ae3fb6b30 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:08.392470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ec4c1-1ce8-49e5-bdaa-488ae3fb6b30 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:08.392470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ec4c1-1ce8-49e5-bdaa-488ae3fb6b30 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:08.394055 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 20:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:28:08.399438 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19608610-2ccc-433a-9fab-c460c2c1542a None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:08.399909 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19608610-2ccc-433a-9fab-c460c2c1542a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:08.400094 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19608610-2ccc-433a-9fab-c460c2c1542a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:08.468213 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-19608610-2ccc-433a-9fab-c460c2c1542a None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:28:08.469435 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-19608610-2ccc-433a-9fab-c460c2c1542a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:08.477663 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:08.572244 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 20:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:28:08.609805 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ec4c1-1ce8-49e5-bdaa-488ae3fb6b30 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:28:08.611357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-425ec4c1-1ce8-49e5-bdaa-488ae3fb6b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:08.738252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ec4c1-1ce8-49e5-bdaa-488ae3fb6b30 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:28:08.798688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:08.823766 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67682a1e-447a-45df-95d4-6047399845a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:28:08.824435 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67682a1e-447a-45df-95d4-6047399845a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:08.824726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67682a1e-447a-45df-95d4-6047399845a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:08.825010 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67682a1e-447a-45df-95d4-6047399845a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:08.826682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-67682a1e-447a-45df-95d4-6047399845a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:08.939791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67682a1e-447a-45df-95d4-6047399845a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:28:08.941284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67682a1e-447a-45df-95d4-6047399845a8 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:28:08.960739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 738/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:28:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:28:11.042837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948429c9-78d4-482b-a496-cdc2e34be2dd None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:11.043464 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948429c9-78d4-482b-a496-cdc2e34be2dd None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:11.043464 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948429c9-78d4-482b-a496-cdc2e34be2dd None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:11.044817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 20:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:28:11.050277 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497bc18-ff1a-441c-8bde-cdca1ea1a0d2 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:11.050781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497bc18-ff1a-441c-8bde-cdca1ea1a0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:11.050894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497bc18-ff1a-441c-8bde-cdca1ea1a0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:11.122897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-3497bc18-ff1a-441c-8bde-cdca1ea1a0d2 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:28:11.124148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3497bc18-ff1a-441c-8bde-cdca1ea1a0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:11.164985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:11.239983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 20:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:28:11.282233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948429c9-78d4-482b-a496-cdc2e34be2dd None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:28:11.283984 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-948429c9-78d4-482b-a496-cdc2e34be2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:11.406758 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948429c9-78d4-482b-a496-cdc2e34be2dd None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:28:11.470274 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:11.529293 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67b5cf-0c6d-4a37-a4b4-f6bc5463dd98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:28:11.529953 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67b5cf-0c6d-4a37-a4b4-f6bc5463dd98 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:11.530237 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67b5cf-0c6d-4a37-a4b4-f6bc5463dd98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:11.530515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67b5cf-0c6d-4a37-a4b4-f6bc5463dd98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:11.532262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5c67b5cf-0c6d-4a37-a4b4-f6bc5463dd98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:11.660177 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c67b5cf-0c6d-4a37-a4b4-f6bc5463dd98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:28:11.661654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c67b5cf-0c6d-4a37-a4b4-f6bc5463dd98 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:28:11.686476 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 725/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:28:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:28:13.869118 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82218cc-9424-4a39-9513-249269e60a87 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:13.869708 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82218cc-9424-4a39-9513-249269e60a87 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:13.869708 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82218cc-9424-4a39-9513-249269e60a87 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:13.871088 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 20:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:28:13.876897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30267f1b-2db2-4068-bde8-422aa47695b3 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:13.878029 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30267f1b-2db2-4068-bde8-422aa47695b3 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:13.878029 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30267f1b-2db2-4068-bde8-422aa47695b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:13.957449 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-30267f1b-2db2-4068-bde8-422aa47695b3 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:28:13.958579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-30267f1b-2db2-4068-bde8-422aa47695b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:13.977745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:14.062727 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 20:28:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:28:14.125749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82218cc-9424-4a39-9513-249269e60a87 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:28:14.128848 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-c82218cc-9424-4a39-9513-249269e60a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:14.263185 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82218cc-9424-4a39-9513-249269e60a87 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:28:14.343692 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:14.354374 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f489881a-2832-47a2-9db3-d7309a241cd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:28:14.355373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489881a-2832-47a2-9db3-d7309a241cd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:14.355685 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489881a-2832-47a2-9db3-d7309a241cd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:14.355995 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489881a-2832-47a2-9db3-d7309a241cd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:14.357800 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f489881a-2832-47a2-9db3-d7309a241cd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:14.490308 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f489881a-2832-47a2-9db3-d7309a241cd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:28:14.492347 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f489881a-2832-47a2-9db3-d7309a241cd7 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:28:14.514910 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 741/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:28:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:28:16.615121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0be77b-7691-4c10-8016-1eb5ffed00e6 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:16.615706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0be77b-7691-4c10-8016-1eb5ffed00e6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:16.615706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0be77b-7691-4c10-8016-1eb5ffed00e6 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:16.617079 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 20:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:28:16.622762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad0fa5-1161-4164-b28d-74c3aa9a7011 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:16.623318 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad0fa5-1161-4164-b28d-74c3aa9a7011 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:16.623435 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad0fa5-1161-4164-b28d-74c3aa9a7011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:16.695219 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-b9ad0fa5-1161-4164-b28d-74c3aa9a7011 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:28:16.696472 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b9ad0fa5-1161-4164-b28d-74c3aa9a7011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:16.748942 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:16.808823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 20:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:28:16.857879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0be77b-7691-4c10-8016-1eb5ffed00e6 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:28:16.860361 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3b0be77b-7691-4c10-8016-1eb5ffed00e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:16.981845 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0be77b-7691-4c10-8016-1eb5ffed00e6 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:28:17.034604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:17.062291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bcde8f-9ed1-431d-9f0b-1982dfb3648c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:28:17.062964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcde8f-9ed1-431d-9f0b-1982dfb3648c service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:17.063252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcde8f-9ed1-431d-9f0b-1982dfb3648c service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:17.063523 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcde8f-9ed1-431d-9f0b-1982dfb3648c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:17.065218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-04bcde8f-9ed1-431d-9f0b-1982dfb3648c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:17.208795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bcde8f-9ed1-431d-9f0b-1982dfb3648c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:28:17.211138 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bcde8f-9ed1-431d-9f0b-1982dfb3648c service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:28:17.233037 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 728/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:28:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:28:26.806047 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=22 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:27.230701 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=66 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:37.368778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161f230-db20-434c-b987-d7c72219d381 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:37.369375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161f230-db20-434c-b987-d7c72219d381 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:37.369375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161f230-db20-434c-b987-d7c72219d381 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:37.371322 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 743/1471] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:28:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:28:37.378281 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6630c6-6ebb-433d-8bf1-7ffa9b827def None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:37.378811 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6630c6-6ebb-433d-8bf1-7ffa9b827def None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:37.379323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6630c6-6ebb-433d-8bf1-7ffa9b827def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:37.462387 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1f6630c6-6ebb-433d-8bf1-7ffa9b827def None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:28:37.463661 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1f6630c6-6ebb-433d-8bf1-7ffa9b827def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:37.574410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 729/1472] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:28:37.633122 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15676df4-40fd-4969-954a-8494e4136c45 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:37.633825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15676df4-40fd-4969-954a-8494e4136c45 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:37.634423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15676df4-40fd-4969-954a-8494e4136c45 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:37.637524 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 744/1473] 10.222.0.52 () {54 vars in 963 bytes} [Thu Jul 14 20:28:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:28:37.646970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55499877-ec45-42f1-9783-7bfcd27d14d7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:37.647273 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55499877-ec45-42f1-9783-7bfcd27d14d7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:37.647545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55499877-ec45-42f1-9783-7bfcd27d14d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:37.737878 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-55499877-ec45-42f1-9783-7bfcd27d14d7 None None] MFA Rules not processed for user `4b7a38b4a7754037a00d11dafc88645b`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:28:37.739453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-55499877-ec45-42f1-9783-7bfcd27d14d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:37.767256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:37.828199 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 730/1474] 10.222.0.52 () {58 vars in 1060 bytes} [Thu Jul 14 20:28:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:28:37.836229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15676df4-40fd-4969-954a-8494e4136c45 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:28:37.837745 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-15676df4-40fd-4969-954a-8494e4136c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:37.961794 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15676df4-40fd-4969-954a-8494e4136c45 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:28:38.048921 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1478c163-f36e-4b82-9297-29f1ab0d3cf2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b7a38b4a7754037a00d11dafc88645b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:28:38.049579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478c163-f36e-4b82-9297-29f1ab0d3cf2 service glance] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:38.049865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478c163-f36e-4b82-9297-29f1ab0d3cf2 service glance] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:38.050141 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478c163-f36e-4b82-9297-29f1ab0d3cf2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:38.051870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-1478c163-f36e-4b82-9297-29f1ab0d3cf2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:38.116145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:38.177700 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1478c163-f36e-4b82-9297-29f1ab0d3cf2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:28:38.179173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1478c163-f36e-4b82-9297-29f1ab0d3cf2 service glance] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:28:38.200869 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 745/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:28:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:28:40.444488 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1e71c-7581-4ad5-8f45-0f2e8f996b75 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:40.445108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1e71c-7581-4ad5-8f45-0f2e8f996b75 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:40.445108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1e71c-7581-4ad5-8f45-0f2e8f996b75 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:40.446455 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 731/1476] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:28:40.452438 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd081589-570c-4469-9ae7-38c5a3e89b07 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:40.453056 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd081589-570c-4469-9ae7-38c5a3e89b07 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:40.453056 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd081589-570c-4469-9ae7-38c5a3e89b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:40.528770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-cd081589-570c-4469-9ae7-38c5a3e89b07 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:28:40.529992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-cd081589-570c-4469-9ae7-38c5a3e89b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:40.647673 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 746/1477] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:28:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:28:40.672314 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1e71c-7581-4ad5-8f45-0f2e8f996b75 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:28:40.673849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-76d1e71c-7581-4ad5-8f45-0f2e8f996b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:40.832706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1e71c-7581-4ad5-8f45-0f2e8f996b75 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:28:40.895330 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:40.952409 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592231c-32e3-4982-b481-491adb65b13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:28:40.953204 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592231c-32e3-4982-b481-491adb65b13b admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:40.953539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592231c-32e3-4982-b481-491adb65b13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:40.953896 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592231c-32e3-4982-b481-491adb65b13b admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:40.954636 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592231c-32e3-4982-b481-491adb65b13b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:28:40.957548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: WARNING py.warnings [None req-2592231c-32e3-4982-b481-491adb65b13b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 20:28:40.957548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: warnings.warn(msg) Jul 14 20:28:40.957548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: Jul 14 20:28:40.958363 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592231c-32e3-4982-b481-491adb65b13b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:28:40.981623 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 732/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 14 20:28:40 2022] GET /identity/v3/projects => generated 2553 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:28:43.167317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861af35-ed31-4f56-8e50-433c50fc7b10 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:43.167925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861af35-ed31-4f56-8e50-433c50fc7b10 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:43.167925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861af35-ed31-4f56-8e50-433c50fc7b10 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:43.169232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 747/1479] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:28:43.176130 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bc6b2-7733-42e9-89cd-d085ff9334dc None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:43.176965 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bc6b2-7733-42e9-89cd-d085ff9334dc None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:43.176965 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bc6b2-7733-42e9-89cd-d085ff9334dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:43.254323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-405bc6b2-7733-42e9-89cd-d085ff9334dc None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:28:43.255597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-405bc6b2-7733-42e9-89cd-d085ff9334dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:43.369081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 733/1480] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:28:43.416043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861af35-ed31-4f56-8e50-433c50fc7b10 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:28:43.418376 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-5861af35-ed31-4f56-8e50-433c50fc7b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:43.430057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:43.609488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861af35-ed31-4f56-8e50-433c50fc7b10 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:28:43.705739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:43.752999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f9410-923c-4786-b6b4-3e39989ce0c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:28:43.753674 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f9410-923c-4786-b6b4-3e39989ce0c0 service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:43.753955 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f9410-923c-4786-b6b4-3e39989ce0c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:43.754230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f9410-923c-4786-b6b4-3e39989ce0c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:43.755954 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f32f9410-923c-4786-b6b4-3e39989ce0c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:43.862768 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f9410-923c-4786-b6b4-3e39989ce0c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:28:43.864465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f9410-923c-4786-b6b4-3e39989ce0c0 service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:28:43.885083 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 748/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:28:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:28:46.187890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09ee55-5274-4e42-b580-7984435ef08a None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:46.188382 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09ee55-5274-4e42-b580-7984435ef08a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:46.188628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09ee55-5274-4e42-b580-7984435ef08a None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:46.190589 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 734/1482] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:28:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:28:46.196021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0162001-5978-4ba5-8698-16788e1bea1d None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:46.196644 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0162001-5978-4ba5-8698-16788e1bea1d None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:46.196644 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0162001-5978-4ba5-8698-16788e1bea1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:46.274641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-e0162001-5978-4ba5-8698-16788e1bea1d None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:28:46.275944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e0162001-5978-4ba5-8698-16788e1bea1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:46.325161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:46.377897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 749/1483] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:28:46.422115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09ee55-5274-4e42-b580-7984435ef08a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:28:46.423705 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-be09ee55-5274-4e42-b580-7984435ef08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:46.449335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:46.540215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09ee55-5274-4e42-b580-7984435ef08a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:28:46.621130 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c048dc-9885-452f-9237-1879d18155ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:28:46.621779 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c048dc-9885-452f-9237-1879d18155ee service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:46.622056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c048dc-9885-452f-9237-1879d18155ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:46.622326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c048dc-9885-452f-9237-1879d18155ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:46.623988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-57c048dc-9885-452f-9237-1879d18155ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:46.704034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:46.741208 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c048dc-9885-452f-9237-1879d18155ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:28:46.742918 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c048dc-9885-452f-9237-1879d18155ee service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:28:46.761218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 735/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:28:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:28:48.873715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330002e4-d7f5-4587-810c-f55260e9a07b None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:48.874305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330002e4-d7f5-4587-810c-f55260e9a07b None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:48.874305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330002e4-d7f5-4587-810c-f55260e9a07b None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:48.875738 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 750/1485] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:28:48.882613 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af491f-189c-4f44-9e64-982f33236cf6 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:48.883184 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af491f-189c-4f44-9e64-982f33236cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:48.883184 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af491f-189c-4f44-9e64-982f33236cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:48.964839 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c6af491f-189c-4f44-9e64-982f33236cf6 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:28:48.966076 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c6af491f-189c-4f44-9e64-982f33236cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:49.075694 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 736/1486] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:28:49.115925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330002e4-d7f5-4587-810c-f55260e9a07b None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:28:49.117485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-330002e4-d7f5-4587-810c-f55260e9a07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:49.234376 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330002e4-d7f5-4587-810c-f55260e9a07b None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:28:49.315221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:49.317344 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180b1c7-06bb-465e-99c3-66143f0cbe7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:28:49.318168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180b1c7-06bb-465e-99c3-66143f0cbe7d service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:49.318706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180b1c7-06bb-465e-99c3-66143f0cbe7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:49.319275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180b1c7-06bb-465e-99c3-66143f0cbe7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:49.321194 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4180b1c7-06bb-465e-99c3-66143f0cbe7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:49.448793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180b1c7-06bb-465e-99c3-66143f0cbe7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:28:49.450249 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180b1c7-06bb-465e-99c3-66143f0cbe7d service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:28:49.471727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 751/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:28:51.895951 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9c4c4-49d8-4caa-8f82-424f8f379511 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:51.896486 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9c4c4-49d8-4caa-8f82-424f8f379511 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:51.896486 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9c4c4-49d8-4caa-8f82-424f8f379511 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:51.898430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 737/1488] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:28:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:28:51.904132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680d39f-e4a5-40a7-bda1-dc7cc5dc6be9 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:51.904655 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680d39f-e4a5-40a7-bda1-dc7cc5dc6be9 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:51.904790 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680d39f-e4a5-40a7-bda1-dc7cc5dc6be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:51.978382 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-5680d39f-e4a5-40a7-bda1-dc7cc5dc6be9 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:28:51.979648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-5680d39f-e4a5-40a7-bda1-dc7cc5dc6be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:52.036258 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:52.085399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 752/1489] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:28:52.128922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9c4c4-49d8-4caa-8f82-424f8f379511 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:28:52.130507 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fdc9c4c4-49d8-4caa-8f82-424f8f379511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:52.171608 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:52.251587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9c4c4-49d8-4caa-8f82-424f8f379511 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:28:52.364597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde63692-8f8d-4b4c-846b-fe3c336dbdf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:28:52.365590 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde63692-8f8d-4b4c-846b-fe3c336dbdf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:52.366117 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde63692-8f8d-4b4c-846b-fe3c336dbdf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:52.366633 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde63692-8f8d-4b4c-846b-fe3c336dbdf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:52.370058 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fde63692-8f8d-4b4c-846b-fe3c336dbdf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:52.483964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:52.521102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde63692-8f8d-4b4c-846b-fe3c336dbdf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:28:52.522562 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde63692-8f8d-4b4c-846b-fe3c336dbdf5 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:28:52.543596 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 738/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:28:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:28:54.881954 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5895a6-deeb-4469-b550-6c2e1c1bf25e None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:54.882773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5895a6-deeb-4469-b550-6c2e1c1bf25e None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:54.882992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5895a6-deeb-4469-b550-6c2e1c1bf25e None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:54.884482 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 753/1491] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 14 20:28:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:28:54.892428 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0dae6-7d1a-4369-be87-310bac35d078 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:54.893005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0dae6-7d1a-4369-be87-310bac35d078 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:54.893005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0dae6-7d1a-4369-be87-310bac35d078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:54.962216 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-81f0dae6-7d1a-4369-be87-310bac35d078 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:28:54.963471 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-81f0dae6-7d1a-4369-be87-310bac35d078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:55.067055 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 739/1492] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 14 20:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:28:55.110448 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5895a6-deeb-4469-b550-6c2e1c1bf25e None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:28:55.112466 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-cb5895a6-deeb-4469-b550-6c2e1c1bf25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:55.223916 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5895a6-deeb-4469-b550-6c2e1c1bf25e None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:28:55.301522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83257722-f054-4f5b-ba31-4258b995457e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:28:55.302181 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83257722-f054-4f5b-ba31-4258b995457e service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:28:55.302464 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83257722-f054-4f5b-ba31-4258b995457e service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:28:55.302743 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83257722-f054-4f5b-ba31-4258b995457e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:28:55.304433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-83257722-f054-4f5b-ba31-4258b995457e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:28:55.314111 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:28:55.429241 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83257722-f054-4f5b-ba31-4258b995457e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:28:55.430611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83257722-f054-4f5b-ba31-4258b995457e service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:28:55.451619 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 754/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:28:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:29:05.065029 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=82 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:29:05.448114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=50 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:30:48.887295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892f287-f0ab-43c7-a0fb-29b2b78c8449 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:30:48.887910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892f287-f0ab-43c7-a0fb-29b2b78c8449 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:30:48.887910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892f287-f0ab-43c7-a0fb-29b2b78c8449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:30:48.978783 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e892f287-f0ab-43c7-a0fb-29b2b78c8449 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:30:48.980032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e892f287-f0ab-43c7-a0fb-29b2b78c8449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:30:49.095708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 740/1494] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:30:49.103739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61267c14-0ee5-4489-a66f-362930fda492 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:30:49.104242 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61267c14-0ee5-4489-a66f-362930fda492 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:30:49.104413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61267c14-0ee5-4489-a66f-362930fda492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:30:49.122981 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-61267c14-0ee5-4489-a66f-362930fda492 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:30:49.124157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-61267c14-0ee5-4489-a66f-362930fda492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:30:49.133204 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 755/1495] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:30:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:30:49.137768 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892f287-f0ab-43c7-a0fb-29b2b78c8449 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:30:49.147739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892f287-f0ab-43c7-a0fb-29b2b78c8449 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:30:49.157665 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7db63-d59b-4e94-b185-fb39cf810479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:30:49.158550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7db63-d59b-4e94-b185-fb39cf810479 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:30:49.158916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7db63-d59b-4e94-b185-fb39cf810479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:30:49.159212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7db63-d59b-4e94-b185-fb39cf810479 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:30:49.159963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7db63-d59b-4e94-b185-fb39cf810479 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:30:49.160272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7db63-d59b-4e94-b185-fb39cf810479 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:30:49.161599 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: WARNING py.warnings [None req-f5a7db63-d59b-4e94-b185-fb39cf810479 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 20:30:49.161599 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: warnings.warn(msg) Jul 14 20:30:49.161599 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: Jul 14 20:30:49.162610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7db63-d59b-4e94-b185-fb39cf810479 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:30:49.177513 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 741/1496] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:30:49.186047 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61267c14-0ee5-4489-a66f-362930fda492 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:30:49.195727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61267c14-0ee5-4489-a66f-362930fda492 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:30:49.200494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ad0eb-6fea-40af-92f0-64304e1f2a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:30:49.201151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ad0eb-6fea-40af-92f0-64304e1f2a3f admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:30:49.201432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ad0eb-6fea-40af-92f0-64304e1f2a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:30:49.201706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ad0eb-6fea-40af-92f0-64304e1f2a3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:30:49.202415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ad0eb-6fea-40af-92f0-64304e1f2a3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:30:49.204281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: WARNING py.warnings [None req-050ad0eb-6fea-40af-92f0-64304e1f2a3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 20:30:49.204281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: warnings.warn(msg) Jul 14 20:30:49.204281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: Jul 14 20:30:49.205478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ad0eb-6fea-40af-92f0-64304e1f2a3f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:30:49.247170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 756/1497] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:30:49 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:30:49.248595 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7db63-d59b-4e94-b185-fb39cf810479 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:30:49.258696 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7db63-d59b-4e94-b185-fb39cf810479 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:30:49.314754 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:30:49.322132 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146623a6-94af-4c47-a474-8ff8b570e61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:30:49.322823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146623a6-94af-4c47-a474-8ff8b570e61f admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:30:49.323141 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146623a6-94af-4c47-a474-8ff8b570e61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:30:49.323418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146623a6-94af-4c47-a474-8ff8b570e61f admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:30:49.324113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146623a6-94af-4c47-a474-8ff8b570e61f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:30:49.325133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: WARNING py.warnings [None req-146623a6-94af-4c47-a474-8ff8b570e61f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 20:30:49.325133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: warnings.warn(msg) Jul 14 20:30:49.325133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: Jul 14 20:30:49.326349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146623a6-94af-4c47-a474-8ff8b570e61f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:30:49.352406 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 742/1498] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:30:49 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:30:49.353834 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ad0eb-6fea-40af-92f0-64304e1f2a3f admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:30:49.365572 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ad0eb-6fea-40af-92f0-64304e1f2a3f admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:30:49.370249 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be99256-5919-458a-9b5e-047f897a8da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:30:49.370927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be99256-5919-458a-9b5e-047f897a8da0 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:30:49.371240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be99256-5919-458a-9b5e-047f897a8da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:30:49.371518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be99256-5919-458a-9b5e-047f897a8da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:30:49.372260 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be99256-5919-458a-9b5e-047f897a8da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:30:49.373295 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: WARNING py.warnings [None req-4be99256-5919-458a-9b5e-047f897a8da0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 20:30:49.373295 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: warnings.warn(msg) Jul 14 20:30:49.373295 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: Jul 14 20:30:49.374182 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be99256-5919-458a-9b5e-047f897a8da0 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:30:49.388126 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 757/1499] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:30:49.391981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146623a6-94af-4c47-a474-8ff8b570e61f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:30:49.403186 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146623a6-94af-4c47-a474-8ff8b570e61f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:30:49.408461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b1e05-2887-45ee-b723-ed0f5e66646a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:30:49.409133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b1e05-2887-45ee-b723-ed0f5e66646a admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:30:49.409439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b1e05-2887-45ee-b723-ed0f5e66646a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:30:49.409711 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b1e05-2887-45ee-b723-ed0f5e66646a admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:30:49.410440 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b1e05-2887-45ee-b723-ed0f5e66646a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:30:49.411487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: WARNING py.warnings [None req-682b1e05-2887-45ee-b723-ed0f5e66646a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 20:30:49.411487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: warnings.warn(msg) Jul 14 20:30:49.411487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: Jul 14 20:30:49.412366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b1e05-2887-45ee-b723-ed0f5e66646a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:30:49.427547 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 743/1500] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:30:49.432436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be99256-5919-458a-9b5e-047f897a8da0 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:30:49.443195 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be99256-5919-458a-9b5e-047f897a8da0 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:30:49.448087 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59967852-949c-4a00-a7fe-9369e82eb9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:30:49.449028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59967852-949c-4a00-a7fe-9369e82eb9d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:30:49.449314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59967852-949c-4a00-a7fe-9369e82eb9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:30:49.449612 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59967852-949c-4a00-a7fe-9369e82eb9d2 admin admin] PATH_INFO: `/v3/projects/28b5934923f144a2aea2c1e631cade9c/users/e323ee06cc46429e845745fb2ee22f78/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:30:49.465761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59967852-949c-4a00-a7fe-9369e82eb9d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28b5934923f144a2aea2c1e631cade9c, user_id=e323ee06cc46429e845745fb2ee22f78, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:30:49.466810 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: WARNING py.warnings [None req-59967852-949c-4a00-a7fe-9369e82eb9d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 20:30:49.466810 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: warnings.warn(msg) Jul 14 20:30:49.466810 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: Jul 14 20:30:49.469290 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59967852-949c-4a00-a7fe-9369e82eb9d2 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:30:49.483891 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 758/1501] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:30:49 2022] PUT /identity/v3/projects/28b5934923f144a2aea2c1e631cade9c/users/e323ee06cc46429e845745fb2ee22f78/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:30:49.496819 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb0029-5a38-4136-bd17-1a9a3321ddb8 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:30:49.497107 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb0029-5a38-4136-bd17-1a9a3321ddb8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:30:49.497388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb0029-5a38-4136-bd17-1a9a3321ddb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:30:49.524988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-caeb0029-5a38-4136-bd17-1a9a3321ddb8 None None] MFA Rules not processed for user `e323ee06cc46429e845745fb2ee22f78`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:30:49.529749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-caeb0029-5a38-4136-bd17-1a9a3321ddb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:30:49.594263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 744/1502] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:30:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:30:49.607911 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb6ecf-ea25-451f-a693-c66e5e84c884 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:30:49.608213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb6ecf-ea25-451f-a693-c66e5e84c884 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:30:49.608494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb6ecf-ea25-451f-a693-c66e5e84c884 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:30:49.609963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 759/1503] 10.222.0.52 () {54 vars in 963 bytes} [Thu Jul 14 20:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:30:49.616753 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4e238-d60a-4c9b-8251-8fab1551bfd1 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:30:49.616753 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4e238-d60a-4c9b-8251-8fab1551bfd1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:30:49.617098 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4e238-d60a-4c9b-8251-8fab1551bfd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:30:49.690030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-16c4e238-d60a-4c9b-8251-8fab1551bfd1 None None] MFA Rules not processed for user `01b4142ea6f142df9b1314532c961f75`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:30:49.695627 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-16c4e238-d60a-4c9b-8251-8fab1551bfd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:30:49.786989 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 745/1504] 10.222.0.52 () {58 vars in 1060 bytes} [Thu Jul 14 20:30:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:30:49.796606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb6ecf-ea25-451f-a693-c66e5e84c884 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:30:49.808209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb6ecf-ea25-451f-a693-c66e5e84c884 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:30:49.811908 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf81f5c-b35d-4c50-8434-5240e967c28e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:30:49.812605 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf81f5c-b35d-4c50-8434-5240e967c28e service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:30:49.812893 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf81f5c-b35d-4c50-8434-5240e967c28e service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:30:49.813173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf81f5c-b35d-4c50-8434-5240e967c28e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:30:49.825078 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf81f5c-b35d-4c50-8434-5240e967c28e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:30:49.828446 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf81f5c-b35d-4c50-8434-5240e967c28e service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:30:49.847054 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 760/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:30:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:30:49.896618 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1faf7-9604-4083-979e-91394523ce76 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:30:49.897129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1faf7-9604-4083-979e-91394523ce76 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:30:49.897237 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1faf7-9604-4083-979e-91394523ce76 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:30:49.898483 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 746/1506] 10.222.0.52 () {52 vars in 829 bytes} [Thu Jul 14 20:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:30:49.902607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf81f5c-b35d-4c50-8434-5240e967c28e service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:30:49.967478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf81f5c-b35d-4c50-8434-5240e967c28e service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:30:49.972265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2986a66-a01e-4666-93cd-c5064d5ba784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:30:49.973158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2986a66-a01e-4666-93cd-c5064d5ba784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:30:49.973439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2986a66-a01e-4666-93cd-c5064d5ba784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:30:49.973715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2986a66-a01e-4666-93cd-c5064d5ba784 admin admin] PATH_INFO: `/v3/users/e323ee06cc46429e845745fb2ee22f78` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:30:49.974966 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2986a66-a01e-4666-93cd-c5064d5ba784 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e323ee06cc46429e845745fb2ee22f78)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:30:49.975988 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: WARNING py.warnings [None req-a2986a66-a01e-4666-93cd-c5064d5ba784 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 20:30:49.975988 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: warnings.warn(msg) Jul 14 20:30:49.975988 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: Jul 14 20:30:49.977196 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2986a66-a01e-4666-93cd-c5064d5ba784 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:30:50.055536 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-a2986a66-a01e-4666-93cd-c5064d5ba784 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e323ee06cc46429e845745fb2ee22f78'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:30:50.068939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-a2986a66-a01e-4666-93cd-c5064d5ba784 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e323ee06cc46429e845745fb2ee22f78'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:30:50.082471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-a2986a66-a01e-4666-93cd-c5064d5ba784 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e323ee06cc46429e845745fb2ee22f78'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:30:50.083224 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-a2986a66-a01e-4666-93cd-c5064d5ba784 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e323ee06cc46429e845745fb2ee22f78'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:30:50.083959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-a2986a66-a01e-4666-93cd-c5064d5ba784 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e323ee06cc46429e845745fb2ee22f78'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:30:50.224190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:30:50.240920 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 761/1507] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:30:49 2022] DELETE /identity/v3/users/e323ee06cc46429e845745fb2ee22f78 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:30:50.247057 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a0563-8989-4740-8622-5a9fde7c1f6a None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:30:50.247365 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a0563-8989-4740-8622-5a9fde7c1f6a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:30:50.247632 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a0563-8989-4740-8622-5a9fde7c1f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:30:50.273510 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ac5a0563-8989-4740-8622-5a9fde7c1f6a None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:30:50.274764 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ac5a0563-8989-4740-8622-5a9fde7c1f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:30:50.285932 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:30:50.346310 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 747/1508] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:30:50.357445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2986a66-a01e-4666-93cd-c5064d5ba784 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:30:50.358888 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-a2986a66-a01e-4666-93cd-c5064d5ba784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:30:50.446008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2986a66-a01e-4666-93cd-c5064d5ba784 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:30:50.449570 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c490e81-babb-4ef2-bdce-8a967f15114e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:30:50.450223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c490e81-babb-4ef2-bdce-8a967f15114e service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:30:50.450500 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c490e81-babb-4ef2-bdce-8a967f15114e service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:30:50.450772 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c490e81-babb-4ef2-bdce-8a967f15114e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:30:50.460351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c490e81-babb-4ef2-bdce-8a967f15114e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:30:50.461786 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c490e81-babb-4ef2-bdce-8a967f15114e service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:30:50.471592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 762/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:30:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:30:50.855143 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a0563-8989-4740-8622-5a9fde7c1f6a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:30:50.856632 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ac5a0563-8989-4740-8622-5a9fde7c1f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:30:50.870064 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a0563-8989-4740-8622-5a9fde7c1f6a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:30:50.874747 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242e8a8-6d3a-49b6-93c3-ce5f621dd76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:30:50.875714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242e8a8-6d3a-49b6-93c3-ce5f621dd76b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:30:50.875951 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242e8a8-6d3a-49b6-93c3-ce5f621dd76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:30:50.876232 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242e8a8-6d3a-49b6-93c3-ce5f621dd76b admin admin] PATH_INFO: `/v3/projects/28b5934923f144a2aea2c1e631cade9c` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:30:50.877480 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242e8a8-6d3a-49b6-93c3-ce5f621dd76b admin admin] RBAC: Authorizing `identity:delete_project(project_id=28b5934923f144a2aea2c1e631cade9c)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:30:50.878540 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: WARNING py.warnings [None req-7242e8a8-6d3a-49b6-93c3-ce5f621dd76b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 20:30:50.878540 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: warnings.warn(msg) Jul 14 20:30:50.878540 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: Jul 14 20:30:50.879755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242e8a8-6d3a-49b6-93c3-ce5f621dd76b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:30:50.953726 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7242e8a8-6d3a-49b6-93c3-ce5f621dd76b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28b5934923f144a2aea2c1e631cade9c', 'request_id': 'req-7242e8a8-6d3a-49b6-93c3-ce5f621dd76b'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:30:50.967785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7242e8a8-6d3a-49b6-93c3-ce5f621dd76b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28b5934923f144a2aea2c1e631cade9c', 'request_id': 'req-7242e8a8-6d3a-49b6-93c3-ce5f621dd76b'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:30:50.974812 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7242e8a8-6d3a-49b6-93c3-ce5f621dd76b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28b5934923f144a2aea2c1e631cade9c', 'request_id': 'req-7242e8a8-6d3a-49b6-93c3-ce5f621dd76b'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:30:50.990177 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7242e8a8-6d3a-49b6-93c3-ce5f621dd76b admin admin] The token cache is being invalidate because project 28b5934923f144a2aea2c1e631cade9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:30:50.990462 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7242e8a8-6d3a-49b6-93c3-ce5f621dd76b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:30:50.991132 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7242e8a8-6d3a-49b6-93c3-ce5f621dd76b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:30:50.992797 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 748/1510] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:30:50 2022] DELETE /identity/v3/projects/28b5934923f144a2aea2c1e631cade9c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:31:00.467273 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:31:00.986668 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:32:32.209228 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318741f9-ed1f-4dfd-ac3a-65638412fa44 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:32.209926 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318741f9-ed1f-4dfd-ac3a-65638412fa44 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:32.209926 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318741f9-ed1f-4dfd-ac3a-65638412fa44 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:32.211372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 20:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:32:32.217294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacb1ea-4d24-4626-ab60-ae4cb5bdae23 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:32.218111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacb1ea-4d24-4626-ab60-ae4cb5bdae23 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:32.218111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacb1ea-4d24-4626-ab60-ae4cb5bdae23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:32.244490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4eacb1ea-4d24-4626-ab60-ae4cb5bdae23 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:32:32.245740 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4eacb1ea-4d24-4626-ab60-ae4cb5bdae23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:32:32.315790 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 20:32:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:32:32.472927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318741f9-ed1f-4dfd-ac3a-65638412fa44 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:32:32.474740 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-318741f9-ed1f-4dfd-ac3a-65638412fa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:32:32.584354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318741f9-ed1f-4dfd-ac3a-65638412fa44 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:32:32.590192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736c606-7f6b-4ed5-b54a-d7262a1eb19d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:32:32.590858 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736c606-7f6b-4ed5-b54a-d7262a1eb19d service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:32.591180 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736c606-7f6b-4ed5-b54a-d7262a1eb19d service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:32.591455 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736c606-7f6b-4ed5-b54a-d7262a1eb19d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:32.607555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a736c606-7f6b-4ed5-b54a-d7262a1eb19d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:32:32.609191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a736c606-7f6b-4ed5-b54a-d7262a1eb19d service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:32:32.622897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 764/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:32:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:32:33.480527 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc499960-f835-4d97-bbde-d2b4590759b7 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:33.481200 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc499960-f835-4d97-bbde-d2b4590759b7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:33.481200 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc499960-f835-4d97-bbde-d2b4590759b7 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:33.482467 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 20:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:32:33.488458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9c6bb-1aec-430c-bd26-65149fa24254 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:33.488908 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9c6bb-1aec-430c-bd26-65149fa24254 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:33.489038 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9c6bb-1aec-430c-bd26-65149fa24254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:33.507777 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-3ff9c6bb-1aec-430c-bd26-65149fa24254 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:32:33.508995 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3ff9c6bb-1aec-430c-bd26-65149fa24254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:32:33.517330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 20:32:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:32:33.677452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc499960-f835-4d97-bbde-d2b4590759b7 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:32:33.682507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c9ae3-19ef-44b1-863e-82730a0f1d3c None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:33.683008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c9ae3-19ef-44b1-863e-82730a0f1d3c None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:33.683439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c9ae3-19ef-44b1-863e-82730a0f1d3c None None] PATH_INFO: `/v3` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:33.685697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 766/1516] 10.222.0.62 () {54 vars in 925 bytes} [Thu Jul 14 20:32:33 2022] GET /identity/v3 => generated 255 bytes in 5 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:32:33.692267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11c7ca-000b-4ed0-9109-02ba6999ea60 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:33.692561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11c7ca-000b-4ed0-9109-02ba6999ea60 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:33.692992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11c7ca-000b-4ed0-9109-02ba6999ea60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:33.693277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc499960-f835-4d97-bbde-d2b4590759b7 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:32:33.702229 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664f984-6aec-4266-aa3e-9881b53435f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:32:33.703292 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664f984-6aec-4266-aa3e-9881b53435f8 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:33.703800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664f984-6aec-4266-aa3e-9881b53435f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:33.704244 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664f984-6aec-4266-aa3e-9881b53435f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:33.715331 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664f984-6aec-4266-aa3e-9881b53435f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:32:33.718133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664f984-6aec-4266-aa3e-9881b53435f8 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:32:33.734677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 751/1517] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:32:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:32:33.763177 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-9c11c7ca-000b-4ed0-9109-02ba6999ea60 None None] MFA Rules not processed for user `1afad68819504c2b864851c6989ae25a`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:32:33.765045 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9c11c7ca-000b-4ed0-9109-02ba6999ea60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:32:33.861042 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 767/1518] 10.222.0.62 () {58 vars in 1015 bytes} [Thu Jul 14 20:32:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:32:42.807454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fce82-c356-4102-a127-b73b6ab656c6 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:42.807989 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fce82-c356-4102-a127-b73b6ab656c6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:42.807989 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fce82-c356-4102-a127-b73b6ab656c6 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:42.809276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 752/1519] 10.222.0.52 () {54 vars in 960 bytes} [Thu Jul 14 20:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:32:42.814435 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c6233-9a93-4865-9262-5eb2eab6d1e3 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:42.815002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c6233-9a93-4865-9262-5eb2eab6d1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:42.815002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c6233-9a93-4865-9262-5eb2eab6d1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:42.836270 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-035c6233-9a93-4865-9262-5eb2eab6d1e3 None None] MFA Rules not processed for user `1afad68819504c2b864851c6989ae25a`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:32:42.837484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-035c6233-9a93-4865-9262-5eb2eab6d1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:32:42.846059 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 768/1520] 10.222.0.52 () {58 vars in 1057 bytes} [Thu Jul 14 20:32:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:32:42.854561 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62fce82-c356-4102-a127-b73b6ab656c6 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:32:42.868344 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62fce82-c356-4102-a127-b73b6ab656c6 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:32:42.873523 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24f7fa-260d-4820-9834-4e3d4bbfc3aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afad68819504c2b864851c6989ae25a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:32:42.874202 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24f7fa-260d-4820-9834-4e3d4bbfc3aa service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:42.874480 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24f7fa-260d-4820-9834-4e3d4bbfc3aa service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:42.874754 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24f7fa-260d-4820-9834-4e3d4bbfc3aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:42.884978 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f24f7fa-260d-4820-9834-4e3d4bbfc3aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:32:42.886765 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f24f7fa-260d-4820-9834-4e3d4bbfc3aa service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:32:42.898544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 753/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:32:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:32:45.022653 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864f921-d671-4925-be05-e33be1dd0afd None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:45.023250 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864f921-d671-4925-be05-e33be1dd0afd None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:45.023250 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864f921-d671-4925-be05-e33be1dd0afd None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:45.024616 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 20:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:32:45.029515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab114d-3594-4771-b725-02dc1a98a9f2 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:45.029912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab114d-3594-4771-b725-02dc1a98a9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:45.030110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab114d-3594-4771-b725-02dc1a98a9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:45.048542 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-09ab114d-3594-4771-b725-02dc1a98a9f2 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:32:45.049746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-09ab114d-3594-4771-b725-02dc1a98a9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:32:45.058297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 20:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:32:45.223145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864f921-d671-4925-be05-e33be1dd0afd None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:32:45.237799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864f921-d671-4925-be05-e33be1dd0afd None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:32:45.243973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84877d96-1b56-4659-8743-7813d4339eef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:32:45.244611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84877d96-1b56-4659-8743-7813d4339eef service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:45.244896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84877d96-1b56-4659-8743-7813d4339eef service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:45.245180 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84877d96-1b56-4659-8743-7813d4339eef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:45.251270 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:32:45.254137 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84877d96-1b56-4659-8743-7813d4339eef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:32:45.255957 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84877d96-1b56-4659-8743-7813d4339eef service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:32:45.265549 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 770/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:32:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:32:46.102259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87bddf-b0f3-4685-890d-15641197a6b2 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:46.102845 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87bddf-b0f3-4685-890d-15641197a6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:46.102845 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87bddf-b0f3-4685-890d-15641197a6b2 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:46.104268 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 20:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:32:46.109969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d146d-ffdf-4a3d-9ed1-8a8043d1c938 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:46.110418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d146d-ffdf-4a3d-9ed1-8a8043d1c938 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:46.111097 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d146d-ffdf-4a3d-9ed1-8a8043d1c938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:46.130833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-315d146d-ffdf-4a3d-9ed1-8a8043d1c938 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:32:46.132074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-315d146d-ffdf-4a3d-9ed1-8a8043d1c938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:32:46.140012 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 20:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:32:46.298494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87bddf-b0f3-4685-890d-15641197a6b2 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:32:46.315068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87bddf-b0f3-4685-890d-15641197a6b2 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:32:46.321577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61e502-33c0-465c-9a74-40385bb9e691 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:32:46.322242 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61e502-33c0-465c-9a74-40385bb9e691 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:46.322531 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61e502-33c0-465c-9a74-40385bb9e691 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:46.322812 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61e502-33c0-465c-9a74-40385bb9e691 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:46.332042 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61e502-33c0-465c-9a74-40385bb9e691 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:32:46.333673 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61e502-33c0-465c-9a74-40385bb9e691 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:32:46.344397 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 756/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:32:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:32:50.173559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27ba4f-54e8-434a-8545-6a91f49c7592 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:50.174439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27ba4f-54e8-434a-8545-6a91f49c7592 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:50.174614 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27ba4f-54e8-434a-8545-6a91f49c7592 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:50.176136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 20:32:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:32:50.182666 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95662fbd-7faf-4440-83ad-d183ce97de8a None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:50.183280 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95662fbd-7faf-4440-83ad-d183ce97de8a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:50.183709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95662fbd-7faf-4440-83ad-d183ce97de8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:50.214099 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-95662fbd-7faf-4440-83ad-d183ce97de8a None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:32:50.215492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-95662fbd-7faf-4440-83ad-d183ce97de8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:32:50.223910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 20:32:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:32:50.262404 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27ba4f-54e8-434a-8545-6a91f49c7592 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:32:50.264177 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-bb27ba4f-54e8-434a-8545-6a91f49c7592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:32:50.322384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27ba4f-54e8-434a-8545-6a91f49c7592 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:32:50.325920 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a6073-43df-4cd9-9f73-a80e0f76aa6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:32:50.326563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a6073-43df-4cd9-9f73-a80e0f76aa6f service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:50.326845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a6073-43df-4cd9-9f73-a80e0f76aa6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:50.327148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a6073-43df-4cd9-9f73-a80e0f76aa6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:50.337508 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a6073-43df-4cd9-9f73-a80e0f76aa6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:32:50.338964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a6073-43df-4cd9-9f73-a80e0f76aa6f service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:32:50.349872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 773/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:32:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:32:52.444246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce8c64-78a6-4969-9a64-5c628ee0badd None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:52.444818 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce8c64-78a6-4969-9a64-5c628ee0badd None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:52.444818 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce8c64-78a6-4969-9a64-5c628ee0badd None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:52.446318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 20:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:32:52.452564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416de6b2-3499-4f8c-bbaa-5529e03faa12 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:52.453035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416de6b2-3499-4f8c-bbaa-5529e03faa12 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:52.453132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416de6b2-3499-4f8c-bbaa-5529e03faa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:52.472998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-416de6b2-3499-4f8c-bbaa-5529e03faa12 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:32:52.474198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-416de6b2-3499-4f8c-bbaa-5529e03faa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:32:52.482144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 20:32:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:32:52.520073 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce8c64-78a6-4969-9a64-5c628ee0badd None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:32:52.531647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce8c64-78a6-4969-9a64-5c628ee0badd None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:32:52.534896 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28572808-f38a-48cf-a610-deac35452fb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:32:52.535568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28572808-f38a-48cf-a610-deac35452fb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:52.535847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28572808-f38a-48cf-a610-deac35452fb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:52.536122 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28572808-f38a-48cf-a610-deac35452fb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:52.545507 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28572808-f38a-48cf-a610-deac35452fb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:32:52.546961 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28572808-f38a-48cf-a610-deac35452fb8 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:32:52.556883 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 759/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:32:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:32:56.263152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663066fc-216f-40f6-a8ee-94ebd911d134 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:56.263792 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663066fc-216f-40f6-a8ee-94ebd911d134 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:56.263792 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663066fc-216f-40f6-a8ee-94ebd911d134 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:56.265103 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 20:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:32:56.270978 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8feaa9-cd18-4def-9a95-346dc59a29ff None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:56.271820 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8feaa9-cd18-4def-9a95-346dc59a29ff None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:56.271820 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8feaa9-cd18-4def-9a95-346dc59a29ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:56.291322 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-3a8feaa9-cd18-4def-9a95-346dc59a29ff None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:32:56.292592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3a8feaa9-cd18-4def-9a95-346dc59a29ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:32:56.301311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 20:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:32:56.617770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663066fc-216f-40f6-a8ee-94ebd911d134 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:32:56.619575 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-663066fc-216f-40f6-a8ee-94ebd911d134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:32:56.686884 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663066fc-216f-40f6-a8ee-94ebd911d134 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:32:56.690501 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0431a-8d29-42ac-a9dd-84b21b866368 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:32:56.691285 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0431a-8d29-42ac-a9dd-84b21b866368 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:56.691573 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0431a-8d29-42ac-a9dd-84b21b866368 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:56.691849 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0431a-8d29-42ac-a9dd-84b21b866368 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:56.701843 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0431a-8d29-42ac-a9dd-84b21b866368 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:32:56.703334 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0431a-8d29-42ac-a9dd-84b21b866368 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:32:56.714354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 776/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:32:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:32:58.796905 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd4b44-1b8a-4f23-9d8f-79767dab7b33 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:58.797716 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd4b44-1b8a-4f23-9d8f-79767dab7b33 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:58.797903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd4b44-1b8a-4f23-9d8f-79767dab7b33 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:58.799420 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 20:32:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:32:58.807006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6d20c-32c0-4b10-a4ac-7433e76a1664 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:58.807949 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6d20c-32c0-4b10-a4ac-7433e76a1664 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:58.807949 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6d20c-32c0-4b10-a4ac-7433e76a1664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:58.827820 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-1da6d20c-32c0-4b10-a4ac-7433e76a1664 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:32:58.829074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-1da6d20c-32c0-4b10-a4ac-7433e76a1664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:32:58.841633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 20:32:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:32:59.161889 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd4b44-1b8a-4f23-9d8f-79767dab7b33 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:32:59.163691 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-97fd4b44-1b8a-4f23-9d8f-79767dab7b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:32:59.178822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd4b44-1b8a-4f23-9d8f-79767dab7b33 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:32:59.182571 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b07a15-b089-467a-8e0b-7166e01a41c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:32:59.183396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b07a15-b089-467a-8e0b-7166e01a41c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:32:59.183680 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b07a15-b089-467a-8e0b-7166e01a41c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:32:59.183954 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b07a15-b089-467a-8e0b-7166e01a41c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:32:59.194013 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b07a15-b089-467a-8e0b-7166e01a41c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:32:59.195512 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b07a15-b089-467a-8e0b-7166e01a41c2 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:32:59.205581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 762/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:32:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:33:08.822025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=63 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:33:09.201086 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=66 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:28.995077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2beb9-b4ec-4008-a619-a2c9b04a9b11 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:28.996211 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2beb9-b4ec-4008-a619-a2c9b04a9b11 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:28.996211 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2beb9-b4ec-4008-a619-a2c9b04a9b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.022763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-90b2beb9-b4ec-4008-a619-a2c9b04a9b11 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:29.024147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-90b2beb9-b4ec-4008-a619-a2c9b04a9b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:29.036993 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 778/1540] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:29.046450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe080729-c6b5-4bb4-a6bd-4c12a41ffab5 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.047089 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe080729-c6b5-4bb4-a6bd-4c12a41ffab5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.047089 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe080729-c6b5-4bb4-a6bd-4c12a41ffab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.088608 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-fe080729-c6b5-4bb4-a6bd-4c12a41ffab5 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:29.090727 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fe080729-c6b5-4bb4-a6bd-4c12a41ffab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:29.118255 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 763/1541] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:29.123743 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2beb9-b4ec-4008-a619-a2c9b04a9b11 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:29.143971 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2beb9-b4ec-4008-a619-a2c9b04a9b11 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:29.168678 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888ed4b-505c-4f80-8851-e9dcd43a39bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:29.169762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888ed4b-505c-4f80-8851-e9dcd43a39bd admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.170380 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888ed4b-505c-4f80-8851-e9dcd43a39bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.170668 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888ed4b-505c-4f80-8851-e9dcd43a39bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.171484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888ed4b-505c-4f80-8851-e9dcd43a39bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:34:29.171794 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888ed4b-505c-4f80-8851-e9dcd43a39bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:29.175493 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: WARNING py.warnings [None req-e888ed4b-505c-4f80-8851-e9dcd43a39bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 20:34:29.175493 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: warnings.warn(msg) Jul 14 20:34:29.175493 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: Jul 14 20:34:29.175493 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888ed4b-505c-4f80-8851-e9dcd43a39bd admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:29.198727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 779/1542] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:34:29.216576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe080729-c6b5-4bb4-a6bd-4c12a41ffab5 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:29.252337 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe080729-c6b5-4bb4-a6bd-4c12a41ffab5 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:29.275742 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fbf8b-7eb3-4719-8afc-d08591c812d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:29.276423 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fbf8b-7eb3-4719-8afc-d08591c812d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.276711 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fbf8b-7eb3-4719-8afc-d08591c812d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.276987 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fbf8b-7eb3-4719-8afc-d08591c812d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.277706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40fbf8b-7eb3-4719-8afc-d08591c812d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:29.278739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: WARNING py.warnings [None req-a40fbf8b-7eb3-4719-8afc-d08591c812d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 20:34:29.278739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: warnings.warn(msg) Jul 14 20:34:29.278739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: Jul 14 20:34:29.279431 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb070e2-a509-4690-864e-c35181b7138d None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.279953 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb070e2-a509-4690-864e-c35181b7138d None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.280083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40fbf8b-7eb3-4719-8afc-d08591c812d0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:29.280559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb070e2-a509-4690-864e-c35181b7138d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.316065 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-acb070e2-a509-4690-864e-c35181b7138d None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:29.318378 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-acb070e2-a509-4690-864e-c35181b7138d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:29.346135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 780/1543] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:29.355691 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d5145-9606-4860-9edd-ed223226aaee None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.356343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d5145-9606-4860-9edd-ed223226aaee None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.356835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d5145-9606-4860-9edd-ed223226aaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.361380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 764/1544] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:34:29 2022] POST /identity/v3/projects => generated 393 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:29.363988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fbf8b-7eb3-4719-8afc-d08591c812d0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:29.387137 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-ed2d5145-9606-4860-9edd-ed223226aaee None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:29.388721 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ed2d5145-9606-4860-9edd-ed223226aaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:29.495795 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 781/1545] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:29.496912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21751f3d-6f3f-4eb8-a485-7984a0de21d5 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.497200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21751f3d-6f3f-4eb8-a485-7984a0de21d5 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.497476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21751f3d-6f3f-4eb8-a485-7984a0de21d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.507432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fbf8b-7eb3-4719-8afc-d08591c812d0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:29.524580 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735c3cf-6109-41eb-809e-70640f6b2b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:29.525708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735c3cf-6109-41eb-809e-70640f6b2b1a admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.526215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735c3cf-6109-41eb-809e-70640f6b2b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.526704 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735c3cf-6109-41eb-809e-70640f6b2b1a admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.527759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735c3cf-6109-41eb-809e-70640f6b2b1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:29.530725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735c3cf-6109-41eb-809e-70640f6b2b1a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:29.553604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-21751f3d-6f3f-4eb8-a485-7984a0de21d5 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:29.557863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-21751f3d-6f3f-4eb8-a485-7984a0de21d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:29.577391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 782/1546] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:29.578752 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39314129-7b61-4849-be0d-337fc95c72f7 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.579256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39314129-7b61-4849-be0d-337fc95c72f7 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.579553 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39314129-7b61-4849-be0d-337fc95c72f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.580803 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 765/1547] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:34:29 2022] POST /identity/v3/users => generated 297 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:34:29.582405 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe5d6f-f89b-4b2c-a680-87589198c881 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.582687 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe5d6f-f89b-4b2c-a680-87589198c881 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.583056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe5d6f-f89b-4b2c-a680-87589198c881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.606002 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-03fe5d6f-f89b-4b2c-a680-87589198c881 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:29.606813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-39314129-7b61-4849-be0d-337fc95c72f7 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:29.607248 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-03fe5d6f-f89b-4b2c-a680-87589198c881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:29.608178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-39314129-7b61-4849-be0d-337fc95c72f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:29.621836 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 766/1548] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:29.622696 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe5d6f-f89b-4b2c-a680-87589198c881 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:29.625228 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 783/1549] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:29.626837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7683613-c00f-4a15-939b-1792a0162efa None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.627411 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7683613-c00f-4a15-939b-1792a0162efa None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.627896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7683613-c00f-4a15-939b-1792a0162efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.646892 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe5d6f-f89b-4b2c-a680-87589198c881 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:29.652323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660190f-a36f-43cc-b932-106c51d6fe4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:29.653111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660190f-a36f-43cc-b932-106c51d6fe4e admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.653476 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660190f-a36f-43cc-b932-106c51d6fe4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.653814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660190f-a36f-43cc-b932-106c51d6fe4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.654620 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660190f-a36f-43cc-b932-106c51d6fe4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:34:29.655032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660190f-a36f-43cc-b932-106c51d6fe4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:29.656728 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660190f-a36f-43cc-b932-106c51d6fe4e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:29.657643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-e7683613-c00f-4a15-939b-1792a0162efa None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:29.659601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e7683613-c00f-4a15-939b-1792a0162efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:29.671585 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 784/1551] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:29.673088 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7683613-c00f-4a15-939b-1792a0162efa None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:29.678490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 767/1551] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:34:29.680484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361dbb0-ba7d-444f-b2eb-1666a5676ca4 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.680857 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361dbb0-ba7d-444f-b2eb-1666a5676ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.681208 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361dbb0-ba7d-444f-b2eb-1666a5676ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.688893 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7683613-c00f-4a15-939b-1792a0162efa None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:29.695341 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8175cd-3b4b-4486-a646-8fa477698c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:29.696723 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8175cd-3b4b-4486-a646-8fa477698c94 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.697234 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8175cd-3b4b-4486-a646-8fa477698c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.697695 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8175cd-3b4b-4486-a646-8fa477698c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.698767 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8175cd-3b4b-4486-a646-8fa477698c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:29.700928 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8175cd-3b4b-4486-a646-8fa477698c94 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:29.701747 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-9361dbb0-ba7d-444f-b2eb-1666a5676ca4 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:29.703098 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9361dbb0-ba7d-444f-b2eb-1666a5676ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:29.711558 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 768/1552] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:29.714234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8deaa8-e88f-4766-af9b-88dcf4c3c846 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.714596 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8deaa8-e88f-4766-af9b-88dcf4c3c846 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.715036 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8deaa8-e88f-4766-af9b-88dcf4c3c846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.717891 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 785/1553] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:29.719737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b384ef-2974-4d8a-930e-882167dd9338 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.720226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b384ef-2974-4d8a-930e-882167dd9338 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.720825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b384ef-2974-4d8a-930e-882167dd9338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.738056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-0a8deaa8-e88f-4766-af9b-88dcf4c3c846 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:29.739950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0a8deaa8-e88f-4766-af9b-88dcf4c3c846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:29.742864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-01b384ef-2974-4d8a-930e-882167dd9338 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:29.744437 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-01b384ef-2974-4d8a-930e-882167dd9338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:29.748827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 769/1554] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:29.750277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250699d2-683c-4625-afde-5fe7a9ee0ce5 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.750649 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250699d2-683c-4625-afde-5fe7a9ee0ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.751131 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250699d2-683c-4625-afde-5fe7a9ee0ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.756739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 786/1555] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:29.760594 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b384ef-2974-4d8a-930e-882167dd9338 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:29.770696 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-250699d2-683c-4625-afde-5fe7a9ee0ce5 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:29.772015 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b384ef-2974-4d8a-930e-882167dd9338 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:29.772963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-250699d2-683c-4625-afde-5fe7a9ee0ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:29.776875 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337271b-63df-4937-a17c-830fb4e9dddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:29.777735 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337271b-63df-4937-a17c-830fb4e9dddc admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.778025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337271b-63df-4937-a17c-830fb4e9dddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.778295 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337271b-63df-4937-a17c-830fb4e9dddc admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.779072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a337271b-63df-4937-a17c-830fb4e9dddc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:29.780990 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a337271b-63df-4937-a17c-830fb4e9dddc admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:29.782594 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 770/1556] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:29.783551 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250699d2-683c-4625-afde-5fe7a9ee0ce5 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:29.799223 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250699d2-683c-4625-afde-5fe7a9ee0ce5 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:29.805147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc32701-4404-4927-9e82-1f92368f5cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:29.806143 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc32701-4404-4927-9e82-1f92368f5cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.806285 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc32701-4404-4927-9e82-1f92368f5cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.806521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc32701-4404-4927-9e82-1f92368f5cb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.807307 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc32701-4404-4927-9e82-1f92368f5cb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:34:29.807795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc32701-4404-4927-9e82-1f92368f5cb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:29.809439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc32701-4404-4927-9e82-1f92368f5cb3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:29.814619 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 787/1557] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:34:29 2022] POST /identity/v3/projects => generated 389 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:29.815516 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337271b-63df-4937-a17c-830fb4e9dddc admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:29.823708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 771/1558] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:34:29.825016 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77800983-5de0-44dc-baa3-83078e1d4978 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.825305 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77800983-5de0-44dc-baa3-83078e1d4978 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.825581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77800983-5de0-44dc-baa3-83078e1d4978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.850608 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-77800983-5de0-44dc-baa3-83078e1d4978 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:29.854970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-77800983-5de0-44dc-baa3-83078e1d4978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:29.892103 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337271b-63df-4937-a17c-830fb4e9dddc admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:29.897835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19234fb-447a-4f45-ba14-47b757d6e47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:29.898618 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19234fb-447a-4f45-ba14-47b757d6e47b admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.898968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19234fb-447a-4f45-ba14-47b757d6e47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.899235 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19234fb-447a-4f45-ba14-47b757d6e47b admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.900152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19234fb-447a-4f45-ba14-47b757d6e47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:29.901721 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19234fb-447a-4f45-ba14-47b757d6e47b admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:29.912674 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 788/1559] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:29.913534 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19234fb-447a-4f45-ba14-47b757d6e47b admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:29.922834 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 772/1560] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:29.923779 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77800983-5de0-44dc-baa3-83078e1d4978 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:29.923936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19234fb-447a-4f45-ba14-47b757d6e47b admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:29.928774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875dfd2c-88b4-42ec-9c9c-96790905af5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:29.930439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875dfd2c-88b4-42ec-9c9c-96790905af5e admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.930735 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875dfd2c-88b4-42ec-9c9c-96790905af5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.931215 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875dfd2c-88b4-42ec-9c9c-96790905af5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.931955 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875dfd2c-88b4-42ec-9c9c-96790905af5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:34:29.932263 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875dfd2c-88b4-42ec-9c9c-96790905af5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:29.933712 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77800983-5de0-44dc-baa3-83078e1d4978 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:29.933856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875dfd2c-88b4-42ec-9c9c-96790905af5e admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:29.939610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc9a32-5c31-485f-93ae-bf1e867f1977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:29.940150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc9a32-5c31-485f-93ae-bf1e867f1977 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.940434 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc9a32-5c31-485f-93ae-bf1e867f1977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.940769 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc9a32-5c31-485f-93ae-bf1e867f1977 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.942003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc9a32-5c31-485f-93ae-bf1e867f1977 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:34:29.942317 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc9a32-5c31-485f-93ae-bf1e867f1977 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:29.943519 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:29.943976 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc9a32-5c31-485f-93ae-bf1e867f1977 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:29.946897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:29.950484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 789/1561] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:34:29.959102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 773/1562] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:34:29.960316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875dfd2c-88b4-42ec-9c9c-96790905af5e admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:29.960587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bf804-e93e-4c46-ba76-f46397e32dde None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.960720 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bf804-e93e-4c46-ba76-f46397e32dde None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.960976 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bf804-e93e-4c46-ba76-f46397e32dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.973102 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875dfd2c-88b4-42ec-9c9c-96790905af5e admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:29.977796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f86b2-8bb6-4539-83f8-4b3767e3365e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:29.978444 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f86b2-8bb6-4539-83f8-4b3767e3365e admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:29.978720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f86b2-8bb6-4539-83f8-4b3767e3365e admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:29.979024 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f86b2-8bb6-4539-83f8-4b3767e3365e admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:29.979154 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f73bf804-e93e-4c46-ba76-f46397e32dde None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:29.979750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f86b2-8bb6-4539-83f8-4b3767e3365e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:34:29.980056 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f86b2-8bb6-4539-83f8-4b3767e3365e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:29.980365 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f73bf804-e93e-4c46-ba76-f46397e32dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:29.981762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f86b2-8bb6-4539-83f8-4b3767e3365e admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:29.989181 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 774/1563] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:29.990113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73bf804-e93e-4c46-ba76-f46397e32dde None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:29.999708 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 790/1564] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:34:30.001343 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73bf804-e93e-4c46-ba76-f46397e32dde None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.001494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f86b2-8bb6-4539-83f8-4b3767e3365e admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.007080 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdf3f5-ec03-4c20-87c4-c736d6b5798b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.007862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdf3f5-ec03-4c20-87c4-c736d6b5798b admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.008293 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdf3f5-ec03-4c20-87c4-c736d6b5798b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.008411 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdf3f5-ec03-4c20-87c4-c736d6b5798b admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.009133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdf3f5-ec03-4c20-87c4-c736d6b5798b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.011283 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdf3f5-ec03-4c20-87c4-c736d6b5798b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.014195 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f86b2-8bb6-4539-83f8-4b3767e3365e admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.019473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8d8d7-59c9-419d-beb0-f8a043fb32df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.020719 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8d8d7-59c9-419d-beb0-f8a043fb32df admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.021010 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8d8d7-59c9-419d-beb0-f8a043fb32df admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.021312 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8d8d7-59c9-419d-beb0-f8a043fb32df admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.023103 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8d8d7-59c9-419d-beb0-f8a043fb32df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.024267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8d8d7-59c9-419d-beb0-f8a043fb32df admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.033563 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 775/1565] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:34:29 2022] POST /identity/v3/users => generated 295 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:30.034379 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdf3f5-ec03-4c20-87c4-c736d6b5798b admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.044825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdf3f5-ec03-4c20-87c4-c736d6b5798b admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.050250 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b7141-fdd5-47f1-b7be-3d7fbb228a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.051197 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b7141-fdd5-47f1-b7be-3d7fbb228a40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.051456 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b7141-fdd5-47f1-b7be-3d7fbb228a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.051887 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b7141-fdd5-47f1-b7be-3d7fbb228a40 admin admin] PATH_INFO: `/v3/projects/cceb4db55bba4bd099104acdcb6d2762/users/765c911957bd4ca68dd520fc4fbba1c9/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.055382 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 791/1566] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:34:30 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:30.057571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a056395-6836-447d-a912-52137f612292 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.058077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a056395-6836-447d-a912-52137f612292 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.058645 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a056395-6836-447d-a912-52137f612292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.069885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b7141-fdd5-47f1-b7be-3d7fbb228a40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cceb4db55bba4bd099104acdcb6d2762, user_id=765c911957bd4ca68dd520fc4fbba1c9, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.070969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: WARNING py.warnings [None req-8b6b7141-fdd5-47f1-b7be-3d7fbb228a40 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 20:34:30.070969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: warnings.warn(msg) Jul 14 20:34:30.070969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: Jul 14 20:34:30.072878 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b7141-fdd5-47f1-b7be-3d7fbb228a40 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.080749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-4a056395-6836-447d-a912-52137f612292 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:30.082912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4a056395-6836-447d-a912-52137f612292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:30.087408 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 776/1567] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:30 2022] PUT /identity/v3/projects/cceb4db55bba4bd099104acdcb6d2762/users/765c911957bd4ca68dd520fc4fbba1c9/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:30.088334 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b7141-fdd5-47f1-b7be-3d7fbb228a40 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.152224 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 792/1568] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:30.152224 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a056395-6836-447d-a912-52137f612292 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.161001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b7141-fdd5-47f1-b7be-3d7fbb228a40 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.167708 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a056395-6836-447d-a912-52137f612292 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.168083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb0473-d664-4b61-aa32-a38168671371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.169179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb0473-d664-4b61-aa32-a38168671371 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.169604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb0473-d664-4b61-aa32-a38168671371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.170069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb0473-d664-4b61-aa32-a38168671371 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.171303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb0473-d664-4b61-aa32-a38168671371 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.174639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92042a35-4157-4c6a-a635-91a18d4aa515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.174779 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb0473-d664-4b61-aa32-a38168671371 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.176203 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042a35-4157-4c6a-a635-91a18d4aa515 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.176661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042a35-4157-4c6a-a635-91a18d4aa515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.177116 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042a35-4157-4c6a-a635-91a18d4aa515 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.178298 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92042a35-4157-4c6a-a635-91a18d4aa515 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.181617 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92042a35-4157-4c6a-a635-91a18d4aa515 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.217709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 777/1569] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:34:30 2022] POST /identity/v3/projects => generated 363 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:30.219944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad45f98-ea5f-47a6-90c3-1e3a49011b09 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.219944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad45f98-ea5f-47a6-90c3-1e3a49011b09 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.219944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad45f98-ea5f-47a6-90c3-1e3a49011b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.233000 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 793/1570] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:34:30 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:30.235316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92042a35-4157-4c6a-a635-91a18d4aa515 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.241147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4ad45f98-ea5f-47a6-90c3-1e3a49011b09 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:30.242362 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4ad45f98-ea5f-47a6-90c3-1e3a49011b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:30.308445 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 778/1571] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:30.312996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad45f98-ea5f-47a6-90c3-1e3a49011b09 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.314553 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92042a35-4157-4c6a-a635-91a18d4aa515 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.320934 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c345e-46fe-4760-9a38-65659b9af924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.321615 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c345e-46fe-4760-9a38-65659b9af924 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.321899 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c345e-46fe-4760-9a38-65659b9af924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.322176 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c345e-46fe-4760-9a38-65659b9af924 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.322881 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c345e-46fe-4760-9a38-65659b9af924 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.325262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad45f98-ea5f-47a6-90c3-1e3a49011b09 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.327050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c345e-46fe-4760-9a38-65659b9af924 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.329338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb91f37-6e90-487a-be2f-e3de6a4df5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.330013 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb91f37-6e90-487a-be2f-e3de6a4df5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.330513 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb91f37-6e90-487a-be2f-e3de6a4df5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.330694 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb91f37-6e90-487a-be2f-e3de6a4df5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.331323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb91f37-6e90-487a-be2f-e3de6a4df5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.332949 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb91f37-6e90-487a-be2f-e3de6a4df5a2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.348703 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 779/1572] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:30.349631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb91f37-6e90-487a-be2f-e3de6a4df5a2 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.355145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 794/1573] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:34:30 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:30.356728 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e98d8-4c19-4816-9463-339b9f4e47e1 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.357008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e98d8-4c19-4816-9463-339b9f4e47e1 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.357279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e98d8-4c19-4816-9463-339b9f4e47e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.360116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb91f37-6e90-487a-be2f-e3de6a4df5a2 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.376565 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-707e98d8-4c19-4816-9463-339b9f4e47e1 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:30.377898 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-707e98d8-4c19-4816-9463-339b9f4e47e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:30.383978 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:30.403089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:30.428292 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb1dad-8e7e-40a3-b147-7270f22e170b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.428986 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb1dad-8e7e-40a3-b147-7270f22e170b admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.429276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb1dad-8e7e-40a3-b147-7270f22e170b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.429555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb1dad-8e7e-40a3-b147-7270f22e170b admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.430271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb1dad-8e7e-40a3-b147-7270f22e170b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.432232 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb1dad-8e7e-40a3-b147-7270f22e170b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.446991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 795/1574] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:30.447864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e98d8-4c19-4816-9463-339b9f4e47e1 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.453602 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 780/1575] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:34:30 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:30.454529 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb1dad-8e7e-40a3-b147-7270f22e170b admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.458501 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e98d8-4c19-4816-9463-339b9f4e47e1 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.463323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acf3d4-296d-4662-9857-c7aef03e78a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.463991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acf3d4-296d-4662-9857-c7aef03e78a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.464271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acf3d4-296d-4662-9857-c7aef03e78a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.464431 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb1dad-8e7e-40a3-b147-7270f22e170b admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.464534 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acf3d4-296d-4662-9857-c7aef03e78a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.465273 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acf3d4-296d-4662-9857-c7aef03e78a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:34:30.465577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acf3d4-296d-4662-9857-c7aef03e78a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.467196 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acf3d4-296d-4662-9857-c7aef03e78a8 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.469200 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601376e-5909-4d44-817d-9a6fd4570688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.469928 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601376e-5909-4d44-817d-9a6fd4570688 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.470220 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601376e-5909-4d44-817d-9a6fd4570688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.470501 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601376e-5909-4d44-817d-9a6fd4570688 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.471264 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601376e-5909-4d44-817d-9a6fd4570688 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.473444 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601376e-5909-4d44-817d-9a6fd4570688 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.480194 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 796/1576] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:34:30.484123 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acf3d4-296d-4662-9857-c7aef03e78a8 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.496090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acf3d4-296d-4662-9857-c7aef03e78a8 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.501196 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b54291-7a60-4b6d-807f-c646b48deefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.501913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b54291-7a60-4b6d-807f-c646b48deefb admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.502371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b54291-7a60-4b6d-807f-c646b48deefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.502806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b54291-7a60-4b6d-807f-c646b48deefb admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.503560 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b54291-7a60-4b6d-807f-c646b48deefb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.504788 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 781/1577] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:34:30 2022] POST /identity/v3/users => generated 282 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:30.505175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: WARNING py.warnings [None req-11b54291-7a60-4b6d-807f-c646b48deefb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 20:34:30.505175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: warnings.warn(msg) Jul 14 20:34:30.505175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: Jul 14 20:34:30.505800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601376e-5909-4d44-817d-9a6fd4570688 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.506442 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b54291-7a60-4b6d-807f-c646b48deefb admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.516095 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601376e-5909-4d44-817d-9a6fd4570688 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.521802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95811dd-b87f-4f19-b7e4-035aed8ade30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.522470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95811dd-b87f-4f19-b7e4-035aed8ade30 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.522753 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95811dd-b87f-4f19-b7e4-035aed8ade30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.523062 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95811dd-b87f-4f19-b7e4-035aed8ade30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.523811 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95811dd-b87f-4f19-b7e4-035aed8ade30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:34:30.524122 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95811dd-b87f-4f19-b7e4-035aed8ade30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.525803 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95811dd-b87f-4f19-b7e4-035aed8ade30 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.528999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 797/1578] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:34:30 2022] POST /identity/v3/users => generated 293 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:30.529835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b54291-7a60-4b6d-807f-c646b48deefb admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.541609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b54291-7a60-4b6d-807f-c646b48deefb admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.546990 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 782/1579] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:34:30.546990 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95811dd-b87f-4f19-b7e4-035aed8ade30 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.550645 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6761fd9-ce79-4012-a3e6-8a48ea1b8b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.551953 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6761fd9-ce79-4012-a3e6-8a48ea1b8b00 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.552238 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6761fd9-ce79-4012-a3e6-8a48ea1b8b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.552513 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6761fd9-ce79-4012-a3e6-8a48ea1b8b00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.553254 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6761fd9-ce79-4012-a3e6-8a48ea1b8b00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.554918 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6761fd9-ce79-4012-a3e6-8a48ea1b8b00 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.556684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95811dd-b87f-4f19-b7e4-035aed8ade30 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.561688 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ed1d5-8fb2-442b-8861-1713861b53af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.562376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ed1d5-8fb2-442b-8861-1713861b53af admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.562727 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ed1d5-8fb2-442b-8861-1713861b53af admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.563100 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ed1d5-8fb2-442b-8861-1713861b53af admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.563870 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ed1d5-8fb2-442b-8861-1713861b53af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.565816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ed1d5-8fb2-442b-8861-1713861b53af admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.567231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 798/1580] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:30.568088 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6761fd9-ce79-4012-a3e6-8a48ea1b8b00 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.579782 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6761fd9-ce79-4012-a3e6-8a48ea1b8b00 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.585121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89442f45-228c-42c1-b00f-740d804177ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.585902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89442f45-228c-42c1-b00f-740d804177ef admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.586192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89442f45-228c-42c1-b00f-740d804177ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.586470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89442f45-228c-42c1-b00f-740d804177ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.587250 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89442f45-228c-42c1-b00f-740d804177ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.588566 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 783/1581] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:34:30 2022] POST /identity/v3/users => generated 284 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:30.588888 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89442f45-228c-42c1-b00f-740d804177ef admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.589409 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ed1d5-8fb2-442b-8861-1713861b53af admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.605512 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 799/1582] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:30.605512 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89442f45-228c-42c1-b00f-740d804177ef admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.612672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89442f45-228c-42c1-b00f-740d804177ef admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.618408 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38276112-54e5-4005-8076-5eb7c2f8b986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.619120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38276112-54e5-4005-8076-5eb7c2f8b986 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.619413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38276112-54e5-4005-8076-5eb7c2f8b986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.619690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38276112-54e5-4005-8076-5eb7c2f8b986 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.620404 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38276112-54e5-4005-8076-5eb7c2f8b986 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.622390 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38276112-54e5-4005-8076-5eb7c2f8b986 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.656446 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 800/1583] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:34:30 2022] POST /identity/v3/projects => generated 357 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:30.659807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38276112-54e5-4005-8076-5eb7c2f8b986 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.666845 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ed1d5-8fb2-442b-8861-1713861b53af admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.725754 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38276112-54e5-4005-8076-5eb7c2f8b986 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.726909 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae80562-2482-4bed-bff7-92c2d97b0f83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.727601 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae80562-2482-4bed-bff7-92c2d97b0f83 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.727887 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae80562-2482-4bed-bff7-92c2d97b0f83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.728163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae80562-2482-4bed-bff7-92c2d97b0f83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.730819 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181982d1-7018-4afd-9442-655df0fe831d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.731675 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181982d1-7018-4afd-9442-655df0fe831d admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.732032 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181982d1-7018-4afd-9442-655df0fe831d admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.732308 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181982d1-7018-4afd-9442-655df0fe831d admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.733032 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181982d1-7018-4afd-9442-655df0fe831d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.734778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181982d1-7018-4afd-9442-655df0fe831d admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.741415 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae80562-2482-4bed-bff7-92c2d97b0f83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.744220 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae80562-2482-4bed-bff7-92c2d97b0f83 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.747656 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 801/1584] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:30.751476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181982d1-7018-4afd-9442-655df0fe831d admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.756752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 784/1585] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:30.759965 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae80562-2482-4bed-bff7-92c2d97b0f83 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.761488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181982d1-7018-4afd-9442-655df0fe831d admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.766366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee23dad-fb1e-4932-85ad-84530660266c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.767083 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee23dad-fb1e-4932-85ad-84530660266c admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.767368 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee23dad-fb1e-4932-85ad-84530660266c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.767644 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee23dad-fb1e-4932-85ad-84530660266c admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.768542 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee23dad-fb1e-4932-85ad-84530660266c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.770210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee23dad-fb1e-4932-85ad-84530660266c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.770398 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae80562-2482-4bed-bff7-92c2d97b0f83 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.776483 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7bcb0-2a5a-4bc6-ab6b-98e064cd7662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.777317 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7bcb0-2a5a-4bc6-ab6b-98e064cd7662 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.777601 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7bcb0-2a5a-4bc6-ab6b-98e064cd7662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.777879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7bcb0-2a5a-4bc6-ab6b-98e064cd7662 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.778695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7bcb0-2a5a-4bc6-ab6b-98e064cd7662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.780557 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7bcb0-2a5a-4bc6-ab6b-98e064cd7662 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.782667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 802/1586] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:30.783530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee23dad-fb1e-4932-85ad-84530660266c admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.800070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee23dad-fb1e-4932-85ad-84530660266c admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.806923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d95acc-81cd-4c59-8d9a-61d12e96a863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.807935 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d95acc-81cd-4c59-8d9a-61d12e96a863 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.808227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d95acc-81cd-4c59-8d9a-61d12e96a863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.808505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d95acc-81cd-4c59-8d9a-61d12e96a863 admin admin] PATH_INFO: `/v3/projects/d8d67813ac3a41d6a7b7ae8ac9629d71/users/5cf43634335a428093ee30c2dd99187b/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.809665 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:30.825734 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 785/1587] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:34:30 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:30.825734 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7bcb0-2a5a-4bc6-ab6b-98e064cd7662 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.827770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d95acc-81cd-4c59-8d9a-61d12e96a863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8d67813ac3a41d6a7b7ae8ac9629d71, user_id=5cf43634335a428093ee30c2dd99187b, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.830862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d95acc-81cd-4c59-8d9a-61d12e96a863 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.852835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 803/1588] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:30 2022] PUT /identity/v3/projects/d8d67813ac3a41d6a7b7ae8ac9629d71/users/5cf43634335a428093ee30c2dd99187b/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:30.852835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d95acc-81cd-4c59-8d9a-61d12e96a863 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.868747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:30.902291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7bcb0-2a5a-4bc6-ab6b-98e064cd7662 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.932835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d95acc-81cd-4c59-8d9a-61d12e96a863 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.944366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d475e76-ad38-4f28-b893-0bed1af0d790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.945076 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d475e76-ad38-4f28-b893-0bed1af0d790 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.945369 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d475e76-ad38-4f28-b893-0bed1af0d790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.945677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d475e76-ad38-4f28-b893-0bed1af0d790 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.946402 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d475e76-ad38-4f28-b893-0bed1af0d790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.948086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d475e76-ad38-4f28-b893-0bed1af0d790 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.968573 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 804/1589] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:30.969592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d475e76-ad38-4f28-b893-0bed1af0d790 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:30.983713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d475e76-ad38-4f28-b893-0bed1af0d790 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:30.988833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce44d2-344b-446e-a2c1-47866d2b70b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.989522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce44d2-344b-446e-a2c1-47866d2b70b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.989806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce44d2-344b-446e-a2c1-47866d2b70b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.990098 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce44d2-344b-446e-a2c1-47866d2b70b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.990862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce44d2-344b-446e-a2c1-47866d2b70b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:30.993166 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce44d2-344b-446e-a2c1-47866d2b70b2 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:30.997290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951fac0-b29a-4056-b15e-136f632abcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:30.998132 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951fac0-b29a-4056-b15e-136f632abcb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:30.998526 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951fac0-b29a-4056-b15e-136f632abcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:30.998910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951fac0-b29a-4056-b15e-136f632abcb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:30.999817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951fac0-b29a-4056-b15e-136f632abcb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:31.003841 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951fac0-b29a-4056-b15e-136f632abcb7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:31.015426 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 786/1590] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:31.016768 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951fac0-b29a-4056-b15e-136f632abcb7 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:31.023822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 805/1591] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:34:30 2022] POST /identity/v3/users => generated 279 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:31.027882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce44d2-344b-446e-a2c1-47866d2b70b2 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:31.032863 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951fac0-b29a-4056-b15e-136f632abcb7 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:31.039394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8306127-e6e2-495d-ab52-287ec0309073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:31.040122 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8306127-e6e2-495d-ab52-287ec0309073 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:31.040374 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce44d2-344b-446e-a2c1-47866d2b70b2 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:31.040524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8306127-e6e2-495d-ab52-287ec0309073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:31.040785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8306127-e6e2-495d-ab52-287ec0309073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:31.041653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8306127-e6e2-495d-ab52-287ec0309073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:31.043598 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8306127-e6e2-495d-ab52-287ec0309073 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:31.046027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd927c7-6e34-4eb7-90be-95eb8f15bff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:31.046892 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd927c7-6e34-4eb7-90be-95eb8f15bff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:31.047198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd927c7-6e34-4eb7-90be-95eb8f15bff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:31.047503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd927c7-6e34-4eb7-90be-95eb8f15bff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:31.048183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd927c7-6e34-4eb7-90be-95eb8f15bff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:31.050390 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd927c7-6e34-4eb7-90be-95eb8f15bff8 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:31.055581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 787/1592] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:31.062597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8306127-e6e2-495d-ab52-287ec0309073 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:31.068142 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 806/1593] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:31.069434 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f9136-0ce5-4a9e-8b4d-d2eec87e4ce9 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:31.069720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f9136-0ce5-4a9e-8b4d-d2eec87e4ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:31.070003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f9136-0ce5-4a9e-8b4d-d2eec87e4ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:31.074301 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8306127-e6e2-495d-ab52-287ec0309073 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:31.080979 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916d671-eb03-4568-88cc-f3f593f2b5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:31.081591 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916d671-eb03-4568-88cc-f3f593f2b5cf admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:31.082418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916d671-eb03-4568-88cc-f3f593f2b5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:31.082706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916d671-eb03-4568-88cc-f3f593f2b5cf admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:31.083461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916d671-eb03-4568-88cc-f3f593f2b5cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:31.085429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916d671-eb03-4568-88cc-f3f593f2b5cf admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:31.110139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 788/1594] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:34:31 2022] POST /identity/v3/users => generated 289 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:31.111141 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-e66f9136-0ce5-4a9e-8b4d-d2eec87e4ce9 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:31.111744 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916d671-eb03-4568-88cc-f3f593f2b5cf admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:31.112580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e66f9136-0ce5-4a9e-8b4d-d2eec87e4ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:31.124945 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916d671-eb03-4568-88cc-f3f593f2b5cf admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:31.126268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 807/1595] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:31.127011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f9136-0ce5-4a9e-8b4d-d2eec87e4ce9 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:31.133538 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d1856-4674-4c95-8617-34456b9b0fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:31.134956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d1856-4674-4c95-8617-34456b9b0fb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:31.135261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d1856-4674-4c95-8617-34456b9b0fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:31.135579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d1856-4674-4c95-8617-34456b9b0fb9 admin admin] PATH_INFO: `/v3/projects/7ccdb0dd0a034f008329f489ed3f7d4d/users/2fdc04cfa44f4a21a1ce9854fd43c068/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:31.140339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f9136-0ce5-4a9e-8b4d-d2eec87e4ce9 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:31.154543 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d1856-4674-4c95-8617-34456b9b0fb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ccdb0dd0a034f008329f489ed3f7d4d, user_id=2fdc04cfa44f4a21a1ce9854fd43c068, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:31.155807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d966409-7a6b-47b2-8cd2-846f57694a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:31.156526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d966409-7a6b-47b2-8cd2-846f57694a46 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:31.156813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d966409-7a6b-47b2-8cd2-846f57694a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:31.157104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d966409-7a6b-47b2-8cd2-846f57694a46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:31.157566 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d1856-4674-4c95-8617-34456b9b0fb9 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:31.157965 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d966409-7a6b-47b2-8cd2-846f57694a46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:31.159663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d966409-7a6b-47b2-8cd2-846f57694a46 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:31.173355 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 808/1596] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:31.174353 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d966409-7a6b-47b2-8cd2-846f57694a46 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:31.185279 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 789/1597] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:31 2022] PUT /identity/v3/projects/7ccdb0dd0a034f008329f489ed3f7d4d/users/2fdc04cfa44f4a21a1ce9854fd43c068/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:31.185279 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d1856-4674-4c95-8617-34456b9b0fb9 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:31.193186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d966409-7a6b-47b2-8cd2-846f57694a46 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:31.253477 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:31.267040 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d1856-4674-4c95-8617-34456b9b0fb9 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:31.278140 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e6b12-718f-44d6-a9c0-0f5d0c069023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:31.279340 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e6b12-718f-44d6-a9c0-0f5d0c069023 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:31.280398 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e6b12-718f-44d6-a9c0-0f5d0c069023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:31.280898 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e6b12-718f-44d6-a9c0-0f5d0c069023 admin admin] PATH_INFO: `/v3/projects/9ca868eabf6b40bf956d00d6a5dca313/users/34d241072914438bb4d09332d01c9831/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:31.301367 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95eac1a-ac86-4a4b-989f-da7e28aa6a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:31.302057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95eac1a-ac86-4a4b-989f-da7e28aa6a28 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:31.302382 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95eac1a-ac86-4a4b-989f-da7e28aa6a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:31.302601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95eac1a-ac86-4a4b-989f-da7e28aa6a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:31.303443 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95eac1a-ac86-4a4b-989f-da7e28aa6a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:31.305141 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95eac1a-ac86-4a4b-989f-da7e28aa6a28 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:31.313099 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e6b12-718f-44d6-a9c0-0f5d0c069023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ca868eabf6b40bf956d00d6a5dca313, user_id=34d241072914438bb4d09332d01c9831, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:31.319429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e6b12-718f-44d6-a9c0-0f5d0c069023 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:31.320941 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 809/1598] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:31.321856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95eac1a-ac86-4a4b-989f-da7e28aa6a28 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:31.339060 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95eac1a-ac86-4a4b-989f-da7e28aa6a28 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:31.347243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 790/1599] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:31 2022] PUT /identity/v3/projects/9ca868eabf6b40bf956d00d6a5dca313/users/34d241072914438bb4d09332d01c9831/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:31.347243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e6b12-718f-44d6-a9c0-0f5d0c069023 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:31.414820 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef921cb-775f-4334-abe3-0e9b81cd7b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:31.415990 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef921cb-775f-4334-abe3-0e9b81cd7b4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:31.416395 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef921cb-775f-4334-abe3-0e9b81cd7b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:31.416688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef921cb-775f-4334-abe3-0e9b81cd7b4d admin admin] PATH_INFO: `/v3/projects/40baf66113cf4fb186ddd5e954a9b7a4/users/16c6975c73954487a1f15649693b1d8a/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:31.421899 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:31.427450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e6b12-718f-44d6-a9c0-0f5d0c069023 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:31.432672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6a845-64d9-45c4-a485-71d0ff721bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:31.433440 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6a845-64d9-45c4-a485-71d0ff721bfe admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:31.433749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6a845-64d9-45c4-a485-71d0ff721bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:31.434542 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6a845-64d9-45c4-a485-71d0ff721bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:31.435578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6a845-64d9-45c4-a485-71d0ff721bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:31.437213 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6a845-64d9-45c4-a485-71d0ff721bfe admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:31.437671 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef921cb-775f-4334-abe3-0e9b81cd7b4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=40baf66113cf4fb186ddd5e954a9b7a4, user_id=16c6975c73954487a1f15649693b1d8a, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:31.440715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef921cb-775f-4334-abe3-0e9b81cd7b4d admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:31.450897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 791/1600] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:31.452287 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6a845-64d9-45c4-a485-71d0ff721bfe admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:31.460640 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 810/1601] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:31 2022] PUT /identity/v3/projects/40baf66113cf4fb186ddd5e954a9b7a4/users/16c6975c73954487a1f15649693b1d8a/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:31.461462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef921cb-775f-4334-abe3-0e9b81cd7b4d admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:31.548793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6a845-64d9-45c4-a485-71d0ff721bfe admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:31.560608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef921cb-775f-4334-abe3-0e9b81cd7b4d admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:31.575847 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f2866-2e52-48aa-86af-d11a80f366e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:31.577288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f2866-2e52-48aa-86af-d11a80f366e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:31.577875 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f2866-2e52-48aa-86af-d11a80f366e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:31.578157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f2866-2e52-48aa-86af-d11a80f366e6 admin admin] PATH_INFO: `/v3/projects/06aff2402ccc40b19528b26bc2c3b1cd/users/fa1fd08767a64255adc4bc427c20d6f7/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:31.603860 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f2866-2e52-48aa-86af-d11a80f366e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06aff2402ccc40b19528b26bc2c3b1cd, user_id=fa1fd08767a64255adc4bc427c20d6f7, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:31.610175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f2866-2e52-48aa-86af-d11a80f366e6 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:31.632098 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 811/1602] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:31 2022] PUT /identity/v3/projects/06aff2402ccc40b19528b26bc2c3b1cd/users/fa1fd08767a64255adc4bc427c20d6f7/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:31.646006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898703aa-17f6-4d54-bff3-cbeb078bc571 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:31.646450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898703aa-17f6-4d54-bff3-cbeb078bc571 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:31.647139 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898703aa-17f6-4d54-bff3-cbeb078bc571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:31.647553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2d675-0bec-4084-9dd2-4ddad8bc3bbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:31.648227 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2d675-0bec-4084-9dd2-4ddad8bc3bbe service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:31.648511 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2d675-0bec-4084-9dd2-4ddad8bc3bbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:31.648783 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2d675-0bec-4084-9dd2-4ddad8bc3bbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:31.669365 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-898703aa-17f6-4d54-bff3-cbeb078bc571 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:31.671400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-898703aa-17f6-4d54-bff3-cbeb078bc571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:31.673339 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2d675-0bec-4084-9dd2-4ddad8bc3bbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:31.676776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2d675-0bec-4084-9dd2-4ddad8bc3bbe service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:31.694879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 792/1603] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:31.695824 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2d675-0bec-4084-9dd2-4ddad8bc3bbe service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:31.706685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2d675-0bec-4084-9dd2-4ddad8bc3bbe service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:31.735631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:31.771359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 812/1604] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:31.773116 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00949977-7e0c-4da6-8964-8022fedf8b65 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:31.773408 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00949977-7e0c-4da6-8964-8022fedf8b65 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:31.773685 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00949977-7e0c-4da6-8964-8022fedf8b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:31.776242 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9607887-5bfe-4b90-917d-3d8b96416f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:31.776931 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9607887-5bfe-4b90-917d-3d8b96416f86 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:31.777215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9607887-5bfe-4b90-917d-3d8b96416f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:31.777491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9607887-5bfe-4b90-917d-3d8b96416f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:31.778234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9607887-5bfe-4b90-917d-3d8b96416f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:31.779939 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9607887-5bfe-4b90-917d-3d8b96416f86 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:31.796430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 793/1605] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:31.801151 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9607887-5bfe-4b90-917d-3d8b96416f86 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:31.807499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-00949977-7e0c-4da6-8964-8022fedf8b65 None None] MFA Rules not processed for user `34d241072914438bb4d09332d01c9831`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:31.809455 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-00949977-7e0c-4da6-8964-8022fedf8b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:31.815308 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9607887-5bfe-4b90-917d-3d8b96416f86 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:31.826700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a8d08-9eb4-4ff8-b765-0e39126058cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:31.828070 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a8d08-9eb4-4ff8-b765-0e39126058cb admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:31.828573 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a8d08-9eb4-4ff8-b765-0e39126058cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:31.829093 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a8d08-9eb4-4ff8-b765-0e39126058cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:31.830088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a8d08-9eb4-4ff8-b765-0e39126058cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:31.833035 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a8d08-9eb4-4ff8-b765-0e39126058cb admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:31.850466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 794/1606] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:31.852452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38ff3f-791b-440e-b371-d522d2ba8dcd None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:31.852822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38ff3f-791b-440e-b371-d522d2ba8dcd None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:31.853177 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38ff3f-791b-440e-b371-d522d2ba8dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:31.877889 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8c38ff3f-791b-440e-b371-d522d2ba8dcd None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:31.880560 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8c38ff3f-791b-440e-b371-d522d2ba8dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:31.895357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 795/1607] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:31.895357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e4dd5-1756-431f-92c4-2398c35aecc1 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:31.895357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e4dd5-1756-431f-92c4-2398c35aecc1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:31.895357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e4dd5-1756-431f-92c4-2398c35aecc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:31.910592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:31.929236 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 813/1608] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:31 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:31.930423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00949977-7e0c-4da6-8964-8022fedf8b65 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:31.932072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-00949977-7e0c-4da6-8964-8022fedf8b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:31.949420 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-7a8e4dd5-1756-431f-92c4-2398c35aecc1 None None] MFA Rules not processed for user `fa1fd08767a64255adc4bc427c20d6f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:31.951172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7a8e4dd5-1756-431f-92c4-2398c35aecc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:32.016050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00949977-7e0c-4da6-8964-8022fedf8b65 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:32.020676 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b6ed3-528e-4937-8778-6a6164f4eb87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:32.021380 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b6ed3-528e-4937-8778-6a6164f4eb87 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:32.022006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b6ed3-528e-4937-8778-6a6164f4eb87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:32.022468 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b6ed3-528e-4937-8778-6a6164f4eb87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:32.038605 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b6ed3-528e-4937-8778-6a6164f4eb87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:32.040188 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b6ed3-528e-4937-8778-6a6164f4eb87 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:32.054177 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 814/1609] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:32.055131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b6ed3-528e-4937-8778-6a6164f4eb87 service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:32.066815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b6ed3-528e-4937-8778-6a6164f4eb87 service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:32.077894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27baf4-8e5e-4a5b-94ae-0499066c8412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:32.078885 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27baf4-8e5e-4a5b-94ae-0499066c8412 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:32.079194 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27baf4-8e5e-4a5b-94ae-0499066c8412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:32.080880 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27baf4-8e5e-4a5b-94ae-0499066c8412 admin admin] PATH_INFO: `/v3/projects/648bdbfd9b0c45349c25e9efae8e259e/users/68a95688d6c34aaeb5afbaae95fbfa3d/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:32.102129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 796/1610] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:31 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:32.103411 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e4dd5-1756-431f-92c4-2398c35aecc1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:32.122108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e4dd5-1756-431f-92c4-2398c35aecc1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:32.136624 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27baf4-8e5e-4a5b-94ae-0499066c8412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=648bdbfd9b0c45349c25e9efae8e259e, user_id=68a95688d6c34aaeb5afbaae95fbfa3d, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:32.139553 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27baf4-8e5e-4a5b-94ae-0499066c8412 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:32.148421 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838b402-3cff-49b4-b045-b71201c87fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:32.150795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838b402-3cff-49b4-b045-b71201c87fce admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:32.151130 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838b402-3cff-49b4-b045-b71201c87fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:32.152685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838b402-3cff-49b4-b045-b71201c87fce admin admin] PATH_INFO: `/v3/projects/c2c8120704e84c6bb83b12980323ff49/users/cc19db89b395477ba352b9c587822405/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:32.162444 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 815/1611] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:32 2022] PUT /identity/v3/projects/648bdbfd9b0c45349c25e9efae8e259e/users/68a95688d6c34aaeb5afbaae95fbfa3d/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:32.163406 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27baf4-8e5e-4a5b-94ae-0499066c8412 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:32.197744 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838b402-3cff-49b4-b045-b71201c87fce admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2c8120704e84c6bb83b12980323ff49, user_id=cc19db89b395477ba352b9c587822405, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:32.202336 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838b402-3cff-49b4-b045-b71201c87fce admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:32.242364 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 797/1612] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:32 2022] PUT /identity/v3/projects/c2c8120704e84c6bb83b12980323ff49/users/cc19db89b395477ba352b9c587822405/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:32.244991 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838b402-3cff-49b4-b045-b71201c87fce admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:32.332264 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27baf4-8e5e-4a5b-94ae-0499066c8412 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:32.361464 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838b402-3cff-49b4-b045-b71201c87fce admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:32.368090 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6152c-f327-494e-b24a-8a378885108f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:32.369268 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6152c-f327-494e-b24a-8a378885108f admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:32.369841 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6152c-f327-494e-b24a-8a378885108f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:32.370217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6152c-f327-494e-b24a-8a378885108f admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:32.371486 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a6152c-f327-494e-b24a-8a378885108f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:32.373827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a6152c-f327-494e-b24a-8a378885108f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:32.407436 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:32.425667 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 798/1613] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:34:32 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:32.425667 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6152c-f327-494e-b24a-8a378885108f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:32.475702 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d68d76-bcd4-414f-93e9-5d301ad6a544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:32.476392 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d68d76-bcd4-414f-93e9-5d301ad6a544 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:32.476674 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d68d76-bcd4-414f-93e9-5d301ad6a544 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:32.477008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d68d76-bcd4-414f-93e9-5d301ad6a544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:32.500804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d68d76-bcd4-414f-93e9-5d301ad6a544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:32.504912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d68d76-bcd4-414f-93e9-5d301ad6a544 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:32.530126 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 816/1614] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:32.533797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa00f3-f53f-4730-b5dc-daca6141320d None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:32.534429 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa00f3-f53f-4730-b5dc-daca6141320d None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:32.534760 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa00f3-f53f-4730-b5dc-daca6141320d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:32.542208 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6152c-f327-494e-b24a-8a378885108f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:32.552692 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8665f8-294c-41fa-ad38-79e171517487 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:32.553441 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8665f8-294c-41fa-ad38-79e171517487 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:32.553738 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8665f8-294c-41fa-ad38-79e171517487 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:32.554018 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8665f8-294c-41fa-ad38-79e171517487 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:32.602081 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-d3fa00f3-f53f-4730-b5dc-daca6141320d None None] MFA Rules not processed for user `68a95688d6c34aaeb5afbaae95fbfa3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:32.606894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d3fa00f3-f53f-4730-b5dc-daca6141320d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:32.673136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:32.718319 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8665f8-294c-41fa-ad38-79e171517487 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:32.719992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8665f8-294c-41fa-ad38-79e171517487 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:32.755227 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 799/1615] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:32 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:32.757871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107132b2-8e26-4e50-a1e7-1285c227276f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:32.758168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107132b2-8e26-4e50-a1e7-1285c227276f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:32.758451 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107132b2-8e26-4e50-a1e7-1285c227276f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:32.785987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 817/1616] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:32 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:32.787047 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa00f3-f53f-4730-b5dc-daca6141320d None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:32.830367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-107132b2-8e26-4e50-a1e7-1285c227276f None None] MFA Rules not processed for user `cc19db89b395477ba352b9c587822405`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:32.832905 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-107132b2-8e26-4e50-a1e7-1285c227276f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:33.046859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa00f3-f53f-4730-b5dc-daca6141320d None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:33.050130 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 800/1617] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:33.050130 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107132b2-8e26-4e50-a1e7-1285c227276f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:33.057958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc40546-e7ce-47dd-bba5-f613d8af3236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:33.058852 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107132b2-8e26-4e50-a1e7-1285c227276f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:33.063534 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc40546-e7ce-47dd-bba5-f613d8af3236 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:33.065480 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc40546-e7ce-47dd-bba5-f613d8af3236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:33.068158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc40546-e7ce-47dd-bba5-f613d8af3236 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:33.069437 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e658e09-dde7-430a-9cb4-c4a42faef2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:33.071536 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e658e09-dde7-430a-9cb4-c4a42faef2a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:33.074386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc40546-e7ce-47dd-bba5-f613d8af3236 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:33.076707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e658e09-dde7-430a-9cb4-c4a42faef2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:33.076707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e658e09-dde7-430a-9cb4-c4a42faef2a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:33.076707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e658e09-dde7-430a-9cb4-c4a42faef2a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:33.082683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e658e09-dde7-430a-9cb4-c4a42faef2a2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:33.107597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc40546-e7ce-47dd-bba5-f613d8af3236 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:33.151725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 801/1618] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:34:33 2022] POST /identity/v3/projects => generated 355 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:33.154391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e658e09-dde7-430a-9cb4-c4a42faef2a2 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:33.203952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 818/1619] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:34:32 2022] POST /identity/v3/users => generated 289 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:33.206108 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc40546-e7ce-47dd-bba5-f613d8af3236 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:33.216491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:33.233288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc40546-e7ce-47dd-bba5-f613d8af3236 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:33.321585 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e658e09-dde7-430a-9cb4-c4a42faef2a2 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:33.334829 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed09929-23d0-40c9-a665-685e8ef3d56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:33.335967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed09929-23d0-40c9-a665-685e8ef3d56f admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:33.335967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed09929-23d0-40c9-a665-685e8ef3d56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:33.336150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed09929-23d0-40c9-a665-685e8ef3d56f admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:33.336808 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed09929-23d0-40c9-a665-685e8ef3d56f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:33.341544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed09929-23d0-40c9-a665-685e8ef3d56f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:33.361512 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94309fd-ad22-4caf-9c63-680f38f14a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:33.362681 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94309fd-ad22-4caf-9c63-680f38f14a80 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:33.364420 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94309fd-ad22-4caf-9c63-680f38f14a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:33.365163 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94309fd-ad22-4caf-9c63-680f38f14a80 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:33.366502 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94309fd-ad22-4caf-9c63-680f38f14a80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:33.370992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94309fd-ad22-4caf-9c63-680f38f14a80 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:33.407938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 802/1620] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:34:33 2022] POST /identity/v3/projects => generated 373 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:33.407938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed09929-23d0-40c9-a665-685e8ef3d56f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:33.436814 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 819/1621] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:34:33 2022] POST /identity/v3/users => generated 284 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:33.441188 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94309fd-ad22-4caf-9c63-680f38f14a80 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:33.490089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94309fd-ad22-4caf-9c63-680f38f14a80 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:33.494452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed09929-23d0-40c9-a665-685e8ef3d56f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:33.499928 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf1edd-7b82-4d2b-af3e-c0af7085cd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:33.500814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf1edd-7b82-4d2b-af3e-c0af7085cd5c admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:33.501120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf1edd-7b82-4d2b-af3e-c0af7085cd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:33.501368 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf1edd-7b82-4d2b-af3e-c0af7085cd5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:33.502188 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf1edd-7b82-4d2b-af3e-c0af7085cd5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:33.505224 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bf1edd-7b82-4d2b-af3e-c0af7085cd5c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:33.505902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2932ea-d8c2-4b7c-a079-81deebdf5baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:33.507599 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2932ea-d8c2-4b7c-a079-81deebdf5baa admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:33.509373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2932ea-d8c2-4b7c-a079-81deebdf5baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:33.510220 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2932ea-d8c2-4b7c-a079-81deebdf5baa admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:33.512969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2932ea-d8c2-4b7c-a079-81deebdf5baa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:33.518110 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2932ea-d8c2-4b7c-a079-81deebdf5baa admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:33.524864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 803/1622] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:33.525824 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf1edd-7b82-4d2b-af3e-c0af7085cd5c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:33.538974 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf1edd-7b82-4d2b-af3e-c0af7085cd5c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:33.546881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d83ce-45e1-49e0-8eb1-86290ad65e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:33.547441 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d83ce-45e1-49e0-8eb1-86290ad65e54 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:33.547734 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d83ce-45e1-49e0-8eb1-86290ad65e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:33.548013 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d83ce-45e1-49e0-8eb1-86290ad65e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:33.549828 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d83ce-45e1-49e0-8eb1-86290ad65e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:33.553245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d83ce-45e1-49e0-8eb1-86290ad65e54 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:33.581214 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 804/1623] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:33.581214 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d83ce-45e1-49e0-8eb1-86290ad65e54 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:33.582314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 820/1624] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:34:33 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:33.585594 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2932ea-d8c2-4b7c-a079-81deebdf5baa admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:33.622367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d83ce-45e1-49e0-8eb1-86290ad65e54 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:33.629235 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2932ea-d8c2-4b7c-a079-81deebdf5baa admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:33.639367 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82134a42-1172-4d5c-afdf-739ab500b33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:33.640864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82134a42-1172-4d5c-afdf-739ab500b33c admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:33.641384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82134a42-1172-4d5c-afdf-739ab500b33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:33.645977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82134a42-1172-4d5c-afdf-739ab500b33c admin admin] PATH_INFO: `/v3/projects/4e0754f3cd9543dbafacf74059719645/users/2c33c3cb623e4ff1b12cb567b89ecd47/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:33.647508 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adf4e7-ccca-4a73-96db-e50c7cf0d371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:33.650293 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adf4e7-ccca-4a73-96db-e50c7cf0d371 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:33.652472 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adf4e7-ccca-4a73-96db-e50c7cf0d371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:33.652786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adf4e7-ccca-4a73-96db-e50c7cf0d371 admin admin] PATH_INFO: `/v3/projects/39e343e224274401b989aaf34ca051da/users/8af066e5013f4c7db97bb880de81b9e5/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:33.671381 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82134a42-1172-4d5c-afdf-739ab500b33c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e0754f3cd9543dbafacf74059719645, user_id=2c33c3cb623e4ff1b12cb567b89ecd47, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:33.680394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82134a42-1172-4d5c-afdf-739ab500b33c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:33.699928 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7adf4e7-ccca-4a73-96db-e50c7cf0d371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39e343e224274401b989aaf34ca051da, user_id=8af066e5013f4c7db97bb880de81b9e5, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:33.704241 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:33.711294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7adf4e7-ccca-4a73-96db-e50c7cf0d371 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:33.720946 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 821/1625] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:33 2022] PUT /identity/v3/projects/4e0754f3cd9543dbafacf74059719645/users/2c33c3cb623e4ff1b12cb567b89ecd47/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:33.720946 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82134a42-1172-4d5c-afdf-739ab500b33c admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:33.755462 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 805/1626] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:33 2022] PUT /identity/v3/projects/39e343e224274401b989aaf34ca051da/users/8af066e5013f4c7db97bb880de81b9e5/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:33.756769 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adf4e7-ccca-4a73-96db-e50c7cf0d371 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:33.910325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82134a42-1172-4d5c-afdf-739ab500b33c admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:33.925344 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adf4e7-ccca-4a73-96db-e50c7cf0d371 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:33.932536 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afffaf9-b1fe-48cc-99ec-fbb2a34fcddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:33.933588 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b0050-05b8-493e-972b-12c281184c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:33.933744 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afffaf9-b1fe-48cc-99ec-fbb2a34fcddb admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:33.934308 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afffaf9-b1fe-48cc-99ec-fbb2a34fcddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:33.934643 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afffaf9-b1fe-48cc-99ec-fbb2a34fcddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:33.935161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b0050-05b8-493e-972b-12c281184c44 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:33.935493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afffaf9-b1fe-48cc-99ec-fbb2a34fcddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:33.935984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b0050-05b8-493e-972b-12c281184c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:33.937421 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afffaf9-b1fe-48cc-99ec-fbb2a34fcddb admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:33.937818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b0050-05b8-493e-972b-12c281184c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:33.940425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b0050-05b8-493e-972b-12c281184c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:33.946426 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b0050-05b8-493e-972b-12c281184c44 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:33.970560 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 806/1627] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:33.972298 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afffaf9-b1fe-48cc-99ec-fbb2a34fcddb admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:33.983209 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:33.989477 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 822/1628] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:33.991351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b0050-05b8-493e-972b-12c281184c44 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:33.999308 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afffaf9-b1fe-48cc-99ec-fbb2a34fcddb admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:34.011037 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d3c68-6e50-49d5-ab88-327559a09296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:34.011744 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d3c68-6e50-49d5-ab88-327559a09296 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:34.012031 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d3c68-6e50-49d5-ab88-327559a09296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:34.012457 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d3c68-6e50-49d5-ab88-327559a09296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:34.013237 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d3c68-6e50-49d5-ab88-327559a09296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:34.014876 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d3c68-6e50-49d5-ab88-327559a09296 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:34.026898 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b0050-05b8-493e-972b-12c281184c44 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:34.039702 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 807/1629] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:34.040713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d3c68-6e50-49d5-ab88-327559a09296 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:34.041526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d7e35-6024-4765-bbd8-a527a6659855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:34.042451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d7e35-6024-4765-bbd8-a527a6659855 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:34.042736 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d7e35-6024-4765-bbd8-a527a6659855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:34.043048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d7e35-6024-4765-bbd8-a527a6659855 admin admin] PATH_INFO: `/v3/projects/4e0754f3cd9543dbafacf74059719645/users/2c33c3cb623e4ff1b12cb567b89ecd47/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:34.045147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d7e35-6024-4765-bbd8-a527a6659855 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e0754f3cd9543dbafacf74059719645, user_id=2c33c3cb623e4ff1b12cb567b89ecd47, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:34.047884 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d7e35-6024-4765-bbd8-a527a6659855 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:34.055129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d3c68-6e50-49d5-ab88-327559a09296 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:34.064462 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9115cd-f0ae-49ec-9574-9a7713b3a11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:34.065670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9115cd-f0ae-49ec-9574-9a7713b3a11d admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:34.066109 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9115cd-f0ae-49ec-9574-9a7713b3a11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:34.067237 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9115cd-f0ae-49ec-9574-9a7713b3a11d admin admin] PATH_INFO: `/v3/projects/4e3c60bbad5a4e72a90e3e014105466e/users/100be68f366948b5ac4ea626ba7ef30a/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:34.074236 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 823/1630] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:33 2022] PUT /identity/v3/projects/4e0754f3cd9543dbafacf74059719645/users/2c33c3cb623e4ff1b12cb567b89ecd47/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:34.084081 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d7e35-6024-4765-bbd8-a527a6659855 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:34.104641 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9115cd-f0ae-49ec-9574-9a7713b3a11d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e3c60bbad5a4e72a90e3e014105466e, user_id=100be68f366948b5ac4ea626ba7ef30a, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:34.108293 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9115cd-f0ae-49ec-9574-9a7713b3a11d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:34.136550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 808/1631] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:34 2022] PUT /identity/v3/projects/4e3c60bbad5a4e72a90e3e014105466e/users/100be68f366948b5ac4ea626ba7ef30a/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:34.138357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f53309-b185-4383-adc4-f4c2b781fd3e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:34.138809 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f53309-b185-4383-adc4-f4c2b781fd3e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:34.139375 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f53309-b185-4383-adc4-f4c2b781fd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:34.204125 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-04f53309-b185-4383-adc4-f4c2b781fd3e None None] MFA Rules not processed for user `2c33c3cb623e4ff1b12cb567b89ecd47`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:34.206570 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-04f53309-b185-4383-adc4-f4c2b781fd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:34.208720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d7e35-6024-4765-bbd8-a527a6659855 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:34.372948 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d346a-b5ef-4835-a40e-06380d1b5184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:34.374050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d346a-b5ef-4835-a40e-06380d1b5184 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:34.374342 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d346a-b5ef-4835-a40e-06380d1b5184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:34.374846 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d346a-b5ef-4835-a40e-06380d1b5184 admin admin] PATH_INFO: `/v3/projects/39e343e224274401b989aaf34ca051da/users/8af066e5013f4c7db97bb880de81b9e5/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:34.380555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d346a-b5ef-4835-a40e-06380d1b5184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39e343e224274401b989aaf34ca051da, user_id=8af066e5013f4c7db97bb880de81b9e5, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:34.382715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d346a-b5ef-4835-a40e-06380d1b5184 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:34.389671 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 809/1632] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:34 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:34.391960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f53309-b185-4383-adc4-f4c2b781fd3e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:34.421686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 824/1633] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:34 2022] PUT /identity/v3/projects/39e343e224274401b989aaf34ca051da/users/8af066e5013f4c7db97bb880de81b9e5/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:34.423627 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d346a-b5ef-4835-a40e-06380d1b5184 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:34.441295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f53309-b185-4383-adc4-f4c2b781fd3e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:34.515050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:34.607481 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d346a-b5ef-4835-a40e-06380d1b5184 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:34.614362 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570401a-368f-4af6-bde9-bdc93fc49e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:34.615222 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570401a-368f-4af6-bde9-bdc93fc49e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:34.615512 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570401a-368f-4af6-bde9-bdc93fc49e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:34.615791 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570401a-368f-4af6-bde9-bdc93fc49e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:34.616547 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570401a-368f-4af6-bde9-bdc93fc49e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:34.618194 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570401a-368f-4af6-bde9-bdc93fc49e25 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:34.621522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91ccf5-e0b7-4893-bc5a-8d6b3f07e9ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:34.622198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91ccf5-e0b7-4893-bc5a-8d6b3f07e9ba service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:34.622483 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91ccf5-e0b7-4893-bc5a-8d6b3f07e9ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:34.622761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91ccf5-e0b7-4893-bc5a-8d6b3f07e9ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:34.663175 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 810/1634] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:34.664992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850078a-3efd-43ac-9bcb-8f0137f1a299 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:34.665489 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850078a-3efd-43ac-9bcb-8f0137f1a299 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:34.665928 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850078a-3efd-43ac-9bcb-8f0137f1a299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:34.743442 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-9850078a-3efd-43ac-9bcb-8f0137f1a299 None None] MFA Rules not processed for user `8af066e5013f4c7db97bb880de81b9e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:34.746760 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9850078a-3efd-43ac-9bcb-8f0137f1a299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:34.827461 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91ccf5-e0b7-4893-bc5a-8d6b3f07e9ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:34.829021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91ccf5-e0b7-4893-bc5a-8d6b3f07e9ba service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:34.854158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 825/1635] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:34 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:34.855069 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91ccf5-e0b7-4893-bc5a-8d6b3f07e9ba service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:34.880327 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91ccf5-e0b7-4893-bc5a-8d6b3f07e9ba service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:34.885956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:34.888986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f7e85-684a-42dd-927c-8f080b08e72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:34.889924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f7e85-684a-42dd-927c-8f080b08e72f admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:34.890221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f7e85-684a-42dd-927c-8f080b08e72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:34.890503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f7e85-684a-42dd-927c-8f080b08e72f admin admin] PATH_INFO: `/v3/projects/4e3c60bbad5a4e72a90e3e014105466e/users/100be68f366948b5ac4ea626ba7ef30a/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:34.905528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f7e85-684a-42dd-927c-8f080b08e72f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e3c60bbad5a4e72a90e3e014105466e, user_id=100be68f366948b5ac4ea626ba7ef30a, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:34.908360 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f7e85-684a-42dd-927c-8f080b08e72f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:34.942589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 826/1636] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:34 2022] PUT /identity/v3/projects/4e3c60bbad5a4e72a90e3e014105466e/users/100be68f366948b5ac4ea626ba7ef30a/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:34.945894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd4050-3469-43f2-838f-5bafd8133cf1 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:34.946194 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd4050-3469-43f2-838f-5bafd8133cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:34.946483 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd4050-3469-43f2-838f-5bafd8133cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:35.014982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-8ccd4050-3469-43f2-838f-5bafd8133cf1 None None] MFA Rules not processed for user `100be68f366948b5ac4ea626ba7ef30a`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:35.016100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-8ccd4050-3469-43f2-838f-5bafd8133cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:35.044653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 811/1637] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:34 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:35.059125 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850078a-3efd-43ac-9bcb-8f0137f1a299 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:35.072935 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850078a-3efd-43ac-9bcb-8f0137f1a299 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:35.155081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15871d-eed1-488f-a5ee-784a86e42e9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:35.156096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15871d-eed1-488f-a5ee-784a86e42e9a service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:35.156577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15871d-eed1-488f-a5ee-784a86e42e9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:35.156939 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15871d-eed1-488f-a5ee-784a86e42e9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:35.193396 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 827/1638] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:35.194327 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd4050-3469-43f2-838f-5bafd8133cf1 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:35.205924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd4050-3469-43f2-838f-5bafd8133cf1 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:35.234539 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11651e7e-57f5-417e-bb0b-9b3221769c48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:35.237876 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11651e7e-57f5-417e-bb0b-9b3221769c48 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:35.238216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11651e7e-57f5-417e-bb0b-9b3221769c48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:35.239840 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11651e7e-57f5-417e-bb0b-9b3221769c48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:35.249781 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15871d-eed1-488f-a5ee-784a86e42e9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:35.251198 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15871d-eed1-488f-a5ee-784a86e42e9a service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:35.279823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 812/1639] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:35 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:34:35.319671 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15871d-eed1-488f-a5ee-784a86e42e9a service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:35.329567 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15871d-eed1-488f-a5ee-784a86e42e9a service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:35.337199 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d6855-e8d9-47cf-80b8-e1a3c27dede7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:35.338759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d6855-e8d9-47cf-80b8-e1a3c27dede7 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:35.339233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d6855-e8d9-47cf-80b8-e1a3c27dede7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:35.339695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d6855-e8d9-47cf-80b8-e1a3c27dede7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:35.408427 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:35.418635 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d6855-e8d9-47cf-80b8-e1a3c27dede7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:35.421812 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d6855-e8d9-47cf-80b8-e1a3c27dede7 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:35.448428 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 813/1640] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:35.453267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11651e7e-57f5-417e-bb0b-9b3221769c48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:35.456345 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11651e7e-57f5-417e-bb0b-9b3221769c48 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:35.491026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 828/1641] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:35 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:36.324694 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d6855-e8d9-47cf-80b8-e1a3c27dede7 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:36.336522 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d6855-e8d9-47cf-80b8-e1a3c27dede7 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:36.349531 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da7f66-2976-4ede-bb7e-06fe58f5d76c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:36.350212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da7f66-2976-4ede-bb7e-06fe58f5d76c service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:36.350496 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da7f66-2976-4ede-bb7e-06fe58f5d76c service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:36.350772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da7f66-2976-4ede-bb7e-06fe58f5d76c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:36.402114 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:36.529044 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da7f66-2976-4ede-bb7e-06fe58f5d76c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:36.530588 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da7f66-2976-4ede-bb7e-06fe58f5d76c service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:36.556474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 814/1642] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:36 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:36.623604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dd33d-867b-4db2-9179-f55ef75d8cb9 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:36.624155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dd33d-867b-4db2-9179-f55ef75d8cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:36.624357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dd33d-867b-4db2-9179-f55ef75d8cb9 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:36.626857 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 829/1643] 10.222.0.52 () {54 vars in 961 bytes} [Thu Jul 14 20:34:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:34:36.634579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb836704-fb8b-48d6-b110-aea0a4e90de6 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:36.635262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb836704-fb8b-48d6-b110-aea0a4e90de6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:36.635782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb836704-fb8b-48d6-b110-aea0a4e90de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:36.689905 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-cb836704-fb8b-48d6-b110-aea0a4e90de6 None None] MFA Rules not processed for user `aa3bb9f1e24a4e9ebd460e97b3736bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:36.692122 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-cb836704-fb8b-48d6-b110-aea0a4e90de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:36.867912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3642415-6428-40ba-8ec5-dc5611878758 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:36.868397 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3642415-6428-40ba-8ec5-dc5611878758 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:36.868920 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3642415-6428-40ba-8ec5-dc5611878758 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:36.871376 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 830/1644] 10.222.0.52 () {54 vars in 912 bytes} [Thu Jul 14 20:34:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:34:36.876308 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 815/1645] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Jul 14 20:34:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:34:36.881162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b742a8-2440-440e-b77f-1e1212f38a94 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:36.881744 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b742a8-2440-440e-b77f-1e1212f38a94 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:36.882268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b742a8-2440-440e-b77f-1e1212f38a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:36.933232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-43b742a8-2440-440e-b77f-1e1212f38a94 None None] MFA Rules not processed for user `aa3bb9f1e24a4e9ebd460e97b3736bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:36.936563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-43b742a8-2440-440e-b77f-1e1212f38a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:36.964090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 831/1646] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 14 20:34:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:36.978660 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb836704-fb8b-48d6-b110-aea0a4e90de6 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:36.993888 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb836704-fb8b-48d6-b110-aea0a4e90de6 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:37.003200 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e999627-8a9d-4e6e-9006-a166413396c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:37.004443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e999627-8a9d-4e6e-9006-a166413396c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:37.004727 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e999627-8a9d-4e6e-9006-a166413396c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:37.004999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e999627-8a9d-4e6e-9006-a166413396c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:37.037517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e999627-8a9d-4e6e-9006-a166413396c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:37.038032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e999627-8a9d-4e6e-9006-a166413396c8 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:37.063193 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 816/1647] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:39.576787 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcd397-882f-47d8-ab02-129f6a0ea26c None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:39.577325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcd397-882f-47d8-ab02-129f6a0ea26c None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:39.577394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcd397-882f-47d8-ab02-129f6a0ea26c None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:39.579610 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 832/1648] 10.222.0.52 () {54 vars in 961 bytes} [Thu Jul 14 20:34:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:34:39.590787 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afe079-bf60-4e6a-a5b7-633e4b33e7a6 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:39.591358 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afe079-bf60-4e6a-a5b7-633e4b33e7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:39.591426 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afe079-bf60-4e6a-a5b7-633e4b33e7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:39.635170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f0afe079-bf60-4e6a-a5b7-633e4b33e7a6 None None] MFA Rules not processed for user `aa3bb9f1e24a4e9ebd460e97b3736bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:39.636628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f0afe079-bf60-4e6a-a5b7-633e4b33e7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:39.666407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 817/1649] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Jul 14 20:34:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:34:39.673004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcaab3-cba0-4449-814f-10b2a19d5710 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:39.673413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcaab3-cba0-4449-814f-10b2a19d5710 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:39.673549 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcaab3-cba0-4449-814f-10b2a19d5710 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:39.676927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 833/1650] 10.222.0.52 () {54 vars in 912 bytes} [Thu Jul 14 20:34:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:34:39.683374 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851fd8c-ea2c-4cf5-8530-99b723f0b34c None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:39.683933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851fd8c-ea2c-4cf5-8530-99b723f0b34c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:39.684495 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851fd8c-ea2c-4cf5-8530-99b723f0b34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:39.732145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-b851fd8c-ea2c-4cf5-8530-99b723f0b34c None None] MFA Rules not processed for user `aa3bb9f1e24a4e9ebd460e97b3736bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:39.733813 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b851fd8c-ea2c-4cf5-8530-99b723f0b34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:39.780082 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 818/1651] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 14 20:34:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:39.795812 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcaab3-cba0-4449-814f-10b2a19d5710 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:39.812895 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcaab3-cba0-4449-814f-10b2a19d5710 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:39.824027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea76d6-8343-4057-9890-d201cb10cbfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:39.825050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea76d6-8343-4057-9890-d201cb10cbfd service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:39.825581 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea76d6-8343-4057-9890-d201cb10cbfd service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:39.826046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea76d6-8343-4057-9890-d201cb10cbfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:39.854285 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea76d6-8343-4057-9890-d201cb10cbfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:39.861271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea76d6-8343-4057-9890-d201cb10cbfd service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:39.904856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 834/1652] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:41.177734 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453adab-5687-4b11-8384-95d3ed17118f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:41.178267 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453adab-5687-4b11-8384-95d3ed17118f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:41.178455 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453adab-5687-4b11-8384-95d3ed17118f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:41.237808 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1453adab-5687-4b11-8384-95d3ed17118f None None] MFA Rules not processed for user `5cf43634335a428093ee30c2dd99187b`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:41.239205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1453adab-5687-4b11-8384-95d3ed17118f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:41.396432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 819/1653] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:41.399772 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea76d6-8343-4057-9890-d201cb10cbfd service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:41.417497 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea76d6-8343-4057-9890-d201cb10cbfd service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:41.431229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54738801-a8d1-449b-9dd0-5dc22bb95503 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:41.433862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54738801-a8d1-449b-9dd0-5dc22bb95503 service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:41.437768 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54738801-a8d1-449b-9dd0-5dc22bb95503 service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:41.438697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54738801-a8d1-449b-9dd0-5dc22bb95503 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:41.452072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54738801-a8d1-449b-9dd0-5dc22bb95503 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:41.453920 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54738801-a8d1-449b-9dd0-5dc22bb95503 service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:41.490316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 835/1654] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:42.717474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453adab-5687-4b11-8384-95d3ed17118f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:42.722371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:42.729945 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453adab-5687-4b11-8384-95d3ed17118f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:42.741244 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbda78-3afd-4393-b539-e2f704354c64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:42.741931 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbda78-3afd-4393-b539-e2f704354c64 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:42.742212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbda78-3afd-4393-b539-e2f704354c64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:42.742560 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbda78-3afd-4393-b539-e2f704354c64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:42.773809 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbda78-3afd-4393-b539-e2f704354c64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:42.778985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbda78-3afd-4393-b539-e2f704354c64 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:42.815026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 820/1655] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:42.860088 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54738801-a8d1-449b-9dd0-5dc22bb95503 service nova] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:43.068296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54738801-a8d1-449b-9dd0-5dc22bb95503 service nova] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:43.081400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d22bf-6532-4a1f-84f0-1cecf9fddef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:43.082916 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d22bf-6532-4a1f-84f0-1cecf9fddef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:43.083142 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d22bf-6532-4a1f-84f0-1cecf9fddef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:43.083423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d22bf-6532-4a1f-84f0-1cecf9fddef5 admin admin] PATH_INFO: `/v3/users/34d241072914438bb4d09332d01c9831` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:43.088542 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d22bf-6532-4a1f-84f0-1cecf9fddef5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34d241072914438bb4d09332d01c9831)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:43.090518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d22bf-6532-4a1f-84f0-1cecf9fddef5 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:43.191317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-4c1d22bf-6532-4a1f-84f0-1cecf9fddef5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34d241072914438bb4d09332d01c9831'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:43.209093 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-4c1d22bf-6532-4a1f-84f0-1cecf9fddef5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34d241072914438bb4d09332d01c9831'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:43.221476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-4c1d22bf-6532-4a1f-84f0-1cecf9fddef5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34d241072914438bb4d09332d01c9831'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:43.228973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-4c1d22bf-6532-4a1f-84f0-1cecf9fddef5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34d241072914438bb4d09332d01c9831'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:43.230089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-4c1d22bf-6532-4a1f-84f0-1cecf9fddef5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34d241072914438bb4d09332d01c9831'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:43.278038 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 836/1656] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:34:42 2022] DELETE /identity/v3/users/34d241072914438bb4d09332d01c9831 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:43.282902 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924ee85-0f2e-433d-8185-b2f02e264e92 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:43.283432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924ee85-0f2e-433d-8185-b2f02e264e92 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:43.284462 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924ee85-0f2e-433d-8185-b2f02e264e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:43.305968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4924ee85-0f2e-433d-8185-b2f02e264e92 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:43.308032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4924ee85-0f2e-433d-8185-b2f02e264e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:43.439257 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 821/1657] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:34:43.478285 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d22bf-6532-4a1f-84f0-1cecf9fddef5 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:43.480357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4c1d22bf-6532-4a1f-84f0-1cecf9fddef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:43.615250 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:43.677720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d22bf-6532-4a1f-84f0-1cecf9fddef5 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:43.692191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6255f-4737-46a0-b4f8-a98dc7ee4b91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:43.695434 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6255f-4737-46a0-b4f8-a98dc7ee4b91 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:43.696208 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6255f-4737-46a0-b4f8-a98dc7ee4b91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:43.697209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6255f-4737-46a0-b4f8-a98dc7ee4b91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:43.730849 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6255f-4737-46a0-b4f8-a98dc7ee4b91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:43.739299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6255f-4737-46a0-b4f8-a98dc7ee4b91 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:43.801662 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 837/1658] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:44.240559 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fff77d-b3cb-4a87-a503-9ff8ff26cd5f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:44.241025 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fff77d-b3cb-4a87-a503-9ff8ff26cd5f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:44.241120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fff77d-b3cb-4a87-a503-9ff8ff26cd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:44.300984 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-79fff77d-b3cb-4a87-a503-9ff8ff26cd5f None None] MFA Rules not processed for user `765c911957bd4ca68dd520fc4fbba1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:44.302717 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-79fff77d-b3cb-4a87-a503-9ff8ff26cd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:44.461533 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 822/1659] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:44.472876 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bf4bc-29e2-4337-99ba-bed4d0f6bef9 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:44.473332 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bf4bc-29e2-4337-99ba-bed4d0f6bef9 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:44.473421 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bf4bc-29e2-4337-99ba-bed4d0f6bef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:44.507011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-507bf4bc-29e2-4337-99ba-bed4d0f6bef9 None None] MFA Rules not processed for user `765c911957bd4ca68dd520fc4fbba1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:44.508385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-507bf4bc-29e2-4337-99ba-bed4d0f6bef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:44.543802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 838/1660] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:44.554903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fff77d-b3cb-4a87-a503-9ff8ff26cd5f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:44.556627 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-79fff77d-b3cb-4a87-a503-9ff8ff26cd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:44.797031 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fff77d-b3cb-4a87-a503-9ff8ff26cd5f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:44.813070 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a6816-5a9e-4232-af69-428710d1eb8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:44.813070 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a6816-5a9e-4232-af69-428710d1eb8c service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:44.815462 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a6816-5a9e-4232-af69-428710d1eb8c service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:44.816687 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a6816-5a9e-4232-af69-428710d1eb8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:44.833516 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed8fd9-606e-40fa-8173-3ad787ad638d None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:44.834081 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed8fd9-606e-40fa-8173-3ad787ad638d None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:44.834617 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed8fd9-606e-40fa-8173-3ad787ad638d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:44.841868 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1a6816-5a9e-4232-af69-428710d1eb8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:44.844446 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1a6816-5a9e-4232-af69-428710d1eb8c service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:44.877686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-64ed8fd9-606e-40fa-8173-3ad787ad638d None None] MFA Rules not processed for user `5cf43634335a428093ee30c2dd99187b`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:44.882115 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-64ed8fd9-606e-40fa-8173-3ad787ad638d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:44.888560 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 823/1661] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:45.100556 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 839/1662] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:34:45.204346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a6816-5a9e-4232-af69-428710d1eb8c service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:45.206217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-bf1a6816-5a9e-4232-af69-428710d1eb8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:45.230029 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:45.235960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a6816-5a9e-4232-af69-428710d1eb8c service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:45.241309 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27a480-adaf-4307-a78b-479617c50c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:45.242453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27a480-adaf-4307-a78b-479617c50c44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:45.242661 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27a480-adaf-4307-a78b-479617c50c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:45.242961 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27a480-adaf-4307-a78b-479617c50c44 admin admin] PATH_INFO: `/v3/projects/9ca868eabf6b40bf956d00d6a5dca313` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:45.244214 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27a480-adaf-4307-a78b-479617c50c44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ca868eabf6b40bf956d00d6a5dca313)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:45.246139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27a480-adaf-4307-a78b-479617c50c44 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:45.336803 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7f27a480-adaf-4307-a78b-479617c50c44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ca868eabf6b40bf956d00d6a5dca313', 'request_id': 'req-7f27a480-adaf-4307-a78b-479617c50c44'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:45.345932 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7f27a480-adaf-4307-a78b-479617c50c44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ca868eabf6b40bf956d00d6a5dca313', 'request_id': 'req-7f27a480-adaf-4307-a78b-479617c50c44'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:45.359525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7f27a480-adaf-4307-a78b-479617c50c44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ca868eabf6b40bf956d00d6a5dca313', 'request_id': 'req-7f27a480-adaf-4307-a78b-479617c50c44'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:45.390911 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7f27a480-adaf-4307-a78b-479617c50c44 admin admin] The token cache is being invalidate because project 9ca868eabf6b40bf956d00d6a5dca313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:34:45.391643 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7f27a480-adaf-4307-a78b-479617c50c44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:45.392799 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7f27a480-adaf-4307-a78b-479617c50c44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:45.425760 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed8fd9-606e-40fa-8173-3ad787ad638d None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:45.426605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 824/1663] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:34:45 2022] DELETE /identity/v3/projects/9ca868eabf6b40bf956d00d6a5dca313 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:45.445335 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed8fd9-606e-40fa-8173-3ad787ad638d None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:45.616320 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84385034-b943-4d4e-9249-be31946db6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:45.617356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84385034-b943-4d4e-9249-be31946db6c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:45.617649 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84385034-b943-4d4e-9249-be31946db6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:45.617931 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84385034-b943-4d4e-9249-be31946db6c7 admin admin] PATH_INFO: `/v3/users/8af066e5013f4c7db97bb880de81b9e5` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:45.621768 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84385034-b943-4d4e-9249-be31946db6c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8af066e5013f4c7db97bb880de81b9e5)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:45.624285 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84385034-b943-4d4e-9249-be31946db6c7 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:45.739105 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-84385034-b943-4d4e-9249-be31946db6c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8af066e5013f4c7db97bb880de81b9e5'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:45.760526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-84385034-b943-4d4e-9249-be31946db6c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8af066e5013f4c7db97bb880de81b9e5'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:45.778505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-84385034-b943-4d4e-9249-be31946db6c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8af066e5013f4c7db97bb880de81b9e5'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:45.779799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-84385034-b943-4d4e-9249-be31946db6c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8af066e5013f4c7db97bb880de81b9e5'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:45.782148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-84385034-b943-4d4e-9249-be31946db6c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8af066e5013f4c7db97bb880de81b9e5'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:45.805719 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:45.864389 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 840/1664] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:34:45 2022] DELETE /identity/v3/users/8af066e5013f4c7db97bb880de81b9e5 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:46.437689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92fc7c-088e-42eb-9e3e-ccf90b426415 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:46.438680 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92fc7c-088e-42eb-9e3e-ccf90b426415 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:46.439186 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92fc7c-088e-42eb-9e3e-ccf90b426415 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:46.441621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 825/1665] 10.222.0.52 () {54 vars in 912 bytes} [Thu Jul 14 20:34:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:34:46.448311 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1208a-b28e-471e-8077-b4b7e09efe84 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:46.448610 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1208a-b28e-471e-8077-b4b7e09efe84 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:46.448883 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1208a-b28e-471e-8077-b4b7e09efe84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:46.530522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-a0f1208a-b28e-471e-8077-b4b7e09efe84 None None] MFA Rules not processed for user `f88257850ee34ad895208ab935b6e643`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:46.532001 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-a0f1208a-b28e-471e-8077-b4b7e09efe84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:46.705312 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 841/1666] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 14 20:34:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:46.720119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92fc7c-088e-42eb-9e3e-ccf90b426415 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:46.722631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2c92fc7c-088e-42eb-9e3e-ccf90b426415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:46.774326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92fc7c-088e-42eb-9e3e-ccf90b426415 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:46.793743 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfd3cb-d180-47fc-8296-42b3df5b5455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:46.794635 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfd3cb-d180-47fc-8296-42b3df5b5455 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:46.794946 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfd3cb-d180-47fc-8296-42b3df5b5455 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:46.795234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfd3cb-d180-47fc-8296-42b3df5b5455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:46.818711 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cfd3cb-d180-47fc-8296-42b3df5b5455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:46.825328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cfd3cb-d180-47fc-8296-42b3df5b5455 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:46.838259 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c7f6b-e127-437f-9370-518a94875478 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:46.839271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c7f6b-e127-437f-9370-518a94875478 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:46.839658 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c7f6b-e127-437f-9370-518a94875478 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:46.841586 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 842/1667] 10.222.0.52 () {54 vars in 912 bytes} [Thu Jul 14 20:34:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:34:46.850384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883bf23-0284-4960-907b-0e4dbae049aa None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:46.850846 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883bf23-0284-4960-907b-0e4dbae049aa None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:46.851274 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883bf23-0284-4960-907b-0e4dbae049aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:46.859332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 826/1668] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:46.914557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-4883bf23-0284-4960-907b-0e4dbae049aa None None] MFA Rules not processed for user `f88257850ee34ad895208ab935b6e643`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:46.919881 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4883bf23-0284-4960-907b-0e4dbae049aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:46.943888 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 843/1669] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 14 20:34:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:34:46.994751 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfd3cb-d180-47fc-8296-42b3df5b5455 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:47.010927 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfd3cb-d180-47fc-8296-42b3df5b5455 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:47.029174 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0318b4-1bd1-40be-9cf5-6747a664eab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:47.030046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0318b4-1bd1-40be-9cf5-6747a664eab0 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:47.030482 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0318b4-1bd1-40be-9cf5-6747a664eab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:47.030893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0318b4-1bd1-40be-9cf5-6747a664eab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:47.078027 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0318b4-1bd1-40be-9cf5-6747a664eab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:47.080122 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0318b4-1bd1-40be-9cf5-6747a664eab0 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:47.105572 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 827/1670] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:47.625146 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883bf23-0284-4960-907b-0e4dbae049aa None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:47.627213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4883bf23-0284-4960-907b-0e4dbae049aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:47.852095 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883bf23-0284-4960-907b-0e4dbae049aa None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:47.864838 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c387a-1752-4616-bc39-94a214ad9207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:47.865736 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c387a-1752-4616-bc39-94a214ad9207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:47.866249 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c387a-1752-4616-bc39-94a214ad9207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:47.866534 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c387a-1752-4616-bc39-94a214ad9207 admin admin] PATH_INFO: `/v3/projects/39e343e224274401b989aaf34ca051da` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:47.868443 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c387a-1752-4616-bc39-94a214ad9207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39e343e224274401b989aaf34ca051da)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:47.872315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: WARNING py.warnings [None req-345c387a-1752-4616-bc39-94a214ad9207 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 20:34:47.872315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: warnings.warn(msg) Jul 14 20:34:47.872315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: Jul 14 20:34:47.873870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c387a-1752-4616-bc39-94a214ad9207 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:48.033978 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-345c387a-1752-4616-bc39-94a214ad9207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39e343e224274401b989aaf34ca051da', 'request_id': 'req-345c387a-1752-4616-bc39-94a214ad9207'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:48.060333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-345c387a-1752-4616-bc39-94a214ad9207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39e343e224274401b989aaf34ca051da', 'request_id': 'req-345c387a-1752-4616-bc39-94a214ad9207'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:48.079096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-345c387a-1752-4616-bc39-94a214ad9207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39e343e224274401b989aaf34ca051da', 'request_id': 'req-345c387a-1752-4616-bc39-94a214ad9207'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:48.097608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-345c387a-1752-4616-bc39-94a214ad9207 admin admin] The token cache is being invalidate because project 39e343e224274401b989aaf34ca051da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:34:48.097763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-345c387a-1752-4616-bc39-94a214ad9207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:48.107209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-345c387a-1752-4616-bc39-94a214ad9207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:48.110732 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 844/1671] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:34:47 2022] DELETE /identity/v3/projects/39e343e224274401b989aaf34ca051da => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:48.117286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec14be-8461-4346-b853-74d9d8761aef None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:48.117799 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec14be-8461-4346-b853-74d9d8761aef None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:48.117889 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec14be-8461-4346-b853-74d9d8761aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:48.188163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8aec14be-8461-4346-b853-74d9d8761aef None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:48.198325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8aec14be-8461-4346-b853-74d9d8761aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:48.374717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb2e7b-be9e-40b5-9f8a-6377e9b3d543 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:48.375032 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb2e7b-be9e-40b5-9f8a-6377e9b3d543 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:48.375362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb2e7b-be9e-40b5-9f8a-6377e9b3d543 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:48.377064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 845/1672] 10.222.0.52 () {54 vars in 960 bytes} [Thu Jul 14 20:34:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:34:48.382747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa730a-58a7-4a90-87e8-7f2c22c50671 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:48.383103 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa730a-58a7-4a90-87e8-7f2c22c50671 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:48.384198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa730a-58a7-4a90-87e8-7f2c22c50671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:48.397943 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 828/1673] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:34:48.413236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229a706-9e8e-4214-be29-97c317ce2c85 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:48.413533 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229a706-9e8e-4214-be29-97c317ce2c85 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:48.413815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229a706-9e8e-4214-be29-97c317ce2c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:48.475695 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-fdfa730a-58a7-4a90-87e8-7f2c22c50671 None None] MFA Rules not processed for user `1afad68819504c2b864851c6989ae25a`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:48.486324 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-fdfa730a-58a7-4a90-87e8-7f2c22c50671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:48.491999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8229a706-9e8e-4214-be29-97c317ce2c85 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:48.494315 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8229a706-9e8e-4214-be29-97c317ce2c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:48.510281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:48.537086 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 829/1674] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:34:48.540366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229a706-9e8e-4214-be29-97c317ce2c85 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:48.548871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8229a706-9e8e-4214-be29-97c317ce2c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:48.748295 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 846/1675] 10.222.0.52 () {58 vars in 1057 bytes} [Thu Jul 14 20:34:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:34:48.749757 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa730a-58a7-4a90-87e8-7f2c22c50671 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:48.775761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa730a-58a7-4a90-87e8-7f2c22c50671 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:48.787669 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21202c56-ec82-469d-bae6-9e9861295c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:48.788595 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21202c56-ec82-469d-bae6-9e9861295c0e admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:48.789219 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21202c56-ec82-469d-bae6-9e9861295c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:48.790367 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21202c56-ec82-469d-bae6-9e9861295c0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:48.791358 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21202c56-ec82-469d-bae6-9e9861295c0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:34:48.791460 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:48.798893 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21202c56-ec82-469d-bae6-9e9861295c0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:48.805193 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21202c56-ec82-469d-bae6-9e9861295c0e admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:48.821525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229a706-9e8e-4214-be29-97c317ce2c85 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:48.836391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede7144-9448-4cb2-b177-a6d1c0f1e54e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afad68819504c2b864851c6989ae25a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:48.839654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede7144-9448-4cb2-b177-a6d1c0f1e54e service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:48.840722 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede7144-9448-4cb2-b177-a6d1c0f1e54e service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:48.841004 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede7144-9448-4cb2-b177-a6d1c0f1e54e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:48.850172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3ede7144-9448-4cb2-b177-a6d1c0f1e54e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:48.872378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede7144-9448-4cb2-b177-a6d1c0f1e54e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:48.875091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede7144-9448-4cb2-b177-a6d1c0f1e54e service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:48.881858 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 847/1676] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:34:48.881858 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef663d-4484-4469-9af6-acb310f8add3 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:48.881858 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef663d-4484-4469-9af6-acb310f8add3 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:48.881858 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef663d-4484-4469-9af6-acb310f8add3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:48.887570 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 830/1677] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:48.889251 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78998c0b-8ebb-44cb-b14c-fa10d7588c53 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:48.889536 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78998c0b-8ebb-44cb-b14c-fa10d7588c53 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:48.889847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78998c0b-8ebb-44cb-b14c-fa10d7588c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:48.938318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-78998c0b-8ebb-44cb-b14c-fa10d7588c53 None None] MFA Rules not processed for user `2fdc04cfa44f4a21a1ce9854fd43c068`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:48.939662 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-78998c0b-8ebb-44cb-b14c-fa10d7588c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:48.943111 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-f6ef663d-4484-4469-9af6-acb310f8add3 None None] MFA Rules not processed for user `765c911957bd4ca68dd520fc4fbba1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:48.944340 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f6ef663d-4484-4469-9af6-acb310f8add3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:49.184858 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 831/1678] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:49.186095 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78998c0b-8ebb-44cb-b14c-fa10d7588c53 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:49.230731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 848/1679] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:49.232155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef663d-4484-4469-9af6-acb310f8add3 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:49.255613 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78998c0b-8ebb-44cb-b14c-fa10d7588c53 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:49.279651 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aaf000-a953-4c5e-8c79-18bd75ecc7db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afad68819504c2b864851c6989ae25a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:49.282520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aaf000-a953-4c5e-8c79-18bd75ecc7db service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:49.283379 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aaf000-a953-4c5e-8c79-18bd75ecc7db service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:49.284248 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aaf000-a953-4c5e-8c79-18bd75ecc7db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:49.297850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef663d-4484-4469-9af6-acb310f8add3 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:49.340091 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041300a-2cd5-4a7c-981c-a945d072a4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:49.340486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041300a-2cd5-4a7c-981c-a945d072a4f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:49.340769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041300a-2cd5-4a7c-981c-a945d072a4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:49.341045 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041300a-2cd5-4a7c-981c-a945d072a4f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:49.341295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aaf000-a953-4c5e-8c79-18bd75ecc7db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:49.341763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041300a-2cd5-4a7c-981c-a945d072a4f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:49.348747 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aaf000-a953-4c5e-8c79-18bd75ecc7db service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:49.349181 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041300a-2cd5-4a7c-981c-a945d072a4f8 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:49.401070 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 832/1680] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:49.414150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac05167-0df3-4082-8651-b33b2b7daa41 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:49.414530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac05167-0df3-4082-8651-b33b2b7daa41 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:49.417284 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac05167-0df3-4082-8651-b33b2b7daa41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:49.483864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 849/1681] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:34:49 2022] POST /identity/v3/projects => generated 387 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:49.484554 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-dac05167-0df3-4082-8651-b33b2b7daa41 None None] MFA Rules not processed for user `2fdc04cfa44f4a21a1ce9854fd43c068`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:49.485281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041300a-2cd5-4a7c-981c-a945d072a4f8 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:49.486554 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dac05167-0df3-4082-8651-b33b2b7daa41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:49.604057 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 833/1682] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:49.624726 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac05167-0df3-4082-8651-b33b2b7daa41 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:49.639938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dac05167-0df3-4082-8651-b33b2b7daa41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:49.699935 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041300a-2cd5-4a7c-981c-a945d072a4f8 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:49.707925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7322bf-4a34-4b36-b4ce-de658547aef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:49.709199 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7322bf-4a34-4b36-b4ce-de658547aef3 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:49.709811 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7322bf-4a34-4b36-b4ce-de658547aef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:49.710346 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7322bf-4a34-4b36-b4ce-de658547aef3 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:49.711338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7322bf-4a34-4b36-b4ce-de658547aef3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:49.714645 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7322bf-4a34-4b36-b4ce-de658547aef3 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:49.757005 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 850/1683] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:34:49 2022] POST /identity/v3/users => generated 294 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:49.764350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7322bf-4a34-4b36-b4ce-de658547aef3 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:49.786038 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7322bf-4a34-4b36-b4ce-de658547aef3 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:49.809819 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3a366-b336-49d0-bee4-1467b368e583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:49.810663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3a366-b336-49d0-bee4-1467b368e583 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:49.811125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3a366-b336-49d0-bee4-1467b368e583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:49.811540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3a366-b336-49d0-bee4-1467b368e583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:49.829262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3a366-b336-49d0-bee4-1467b368e583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:49.845795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:49.857292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3a366-b336-49d0-bee4-1467b368e583 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:49.860757 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:49.863316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac05167-0df3-4082-8651-b33b2b7daa41 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:49.882274 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c1fda-958a-4906-ab67-2a4de85c08ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:49.883328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c1fda-958a-4906-ab67-2a4de85c08ea service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:49.883817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c1fda-958a-4906-ab67-2a4de85c08ea service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:49.884285 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c1fda-958a-4906-ab67-2a4de85c08ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:49.888340 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 851/1684] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:49.892423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3a366-b336-49d0-bee4-1467b368e583 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:49.916399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c1fda-958a-4906-ab67-2a4de85c08ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:49.918960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c1fda-958a-4906-ab67-2a4de85c08ea service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:49.928378 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3a366-b336-49d0-bee4-1467b368e583 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:49.943128 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e791e4-88f7-4def-9749-ed800cba09ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:49.944054 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e791e4-88f7-4def-9749-ed800cba09ad admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:49.944474 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e791e4-88f7-4def-9749-ed800cba09ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:49.944891 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e791e4-88f7-4def-9749-ed800cba09ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:49.947827 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e791e4-88f7-4def-9749-ed800cba09ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:49.950078 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e791e4-88f7-4def-9749-ed800cba09ad admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:49.953048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 834/1685] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:34:49.990392 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 852/1686] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:49.996018 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c1fda-958a-4906-ab67-2a4de85c08ea service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:50.026821 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c1fda-958a-4906-ab67-2a4de85c08ea service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:50.035627 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7690032-ce90-4ca4-8ce3-a87c9b2819d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:50.036718 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7690032-ce90-4ca4-8ce3-a87c9b2819d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:50.037146 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7690032-ce90-4ca4-8ce3-a87c9b2819d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:50.037560 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7690032-ce90-4ca4-8ce3-a87c9b2819d5 admin admin] PATH_INFO: `/v3/projects/ea74e7cf74ba465ea46af5315466e5b4/users/b75303d11a5f4f9b9742c97faade1613/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:50.116590 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7690032-ce90-4ca4-8ce3-a87c9b2819d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea74e7cf74ba465ea46af5315466e5b4, user_id=b75303d11a5f4f9b9742c97faade1613, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:50.119597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7690032-ce90-4ca4-8ce3-a87c9b2819d5 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:50.173493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 835/1687] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:49 2022] PUT /identity/v3/projects/ea74e7cf74ba465ea46af5315466e5b4/users/b75303d11a5f4f9b9742c97faade1613/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:50.186523 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d592a1f-f53f-48dc-8496-ecb6e8ba99f0 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:50.187016 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d592a1f-f53f-48dc-8496-ecb6e8ba99f0 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:50.187415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d592a1f-f53f-48dc-8496-ecb6e8ba99f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:50.300717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-6d592a1f-f53f-48dc-8496-ecb6e8ba99f0 None None] MFA Rules not processed for user `b75303d11a5f4f9b9742c97faade1613`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:50.304102 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6d592a1f-f53f-48dc-8496-ecb6e8ba99f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:50.533640 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 853/1688] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:50.538320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7690032-ce90-4ca4-8ce3-a87c9b2819d5 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:50.586084 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d592a1f-f53f-48dc-8496-ecb6e8ba99f0 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:50.588447 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6d592a1f-f53f-48dc-8496-ecb6e8ba99f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:50.841661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d592a1f-f53f-48dc-8496-ecb6e8ba99f0 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:50.852838 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c66d99-45b2-476b-942e-cb23102e8ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:50.854390 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c66d99-45b2-476b-942e-cb23102e8ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:50.861421 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c66d99-45b2-476b-942e-cb23102e8ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:50.862321 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c66d99-45b2-476b-942e-cb23102e8ac9 admin admin] PATH_INFO: `/v3/users/fa1fd08767a64255adc4bc427c20d6f7` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:50.866090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c66d99-45b2-476b-942e-cb23102e8ac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa1fd08767a64255adc4bc427c20d6f7)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:50.870788 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7690032-ce90-4ca4-8ce3-a87c9b2819d5 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:50.890805 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c66d99-45b2-476b-942e-cb23102e8ac9 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:50.895568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419000af-2e2a-4665-9072-21b90801a90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:50.896372 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419000af-2e2a-4665-9072-21b90801a90a admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:50.910021 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419000af-2e2a-4665-9072-21b90801a90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:50.910021 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419000af-2e2a-4665-9072-21b90801a90a admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:50.910021 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419000af-2e2a-4665-9072-21b90801a90a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:50.934809 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419000af-2e2a-4665-9072-21b90801a90a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:51.026597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 836/1689] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:34:50 2022] POST /identity/v3/projects => generated 387 bytes in 489 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:51.030806 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419000af-2e2a-4665-9072-21b90801a90a admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:51.083781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-14c66d99-45b2-476b-942e-cb23102e8ac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa1fd08767a64255adc4bc427c20d6f7'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:51.104898 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-14c66d99-45b2-476b-942e-cb23102e8ac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa1fd08767a64255adc4bc427c20d6f7'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:51.127069 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-14c66d99-45b2-476b-942e-cb23102e8ac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa1fd08767a64255adc4bc427c20d6f7'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:51.134395 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-14c66d99-45b2-476b-942e-cb23102e8ac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa1fd08767a64255adc4bc427c20d6f7'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:51.135036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-14c66d99-45b2-476b-942e-cb23102e8ac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa1fd08767a64255adc4bc427c20d6f7'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:51.164505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:51.186400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 854/1690] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:34:50 2022] DELETE /identity/v3/users/fa1fd08767a64255adc4bc427c20d6f7 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:34:51.201489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2adbf-d9cc-411b-94ad-22f1c6e503c6 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:51.201972 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2adbf-d9cc-411b-94ad-22f1c6e503c6 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:51.202352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2adbf-d9cc-411b-94ad-22f1c6e503c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:51.254808 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-11a2adbf-d9cc-411b-94ad-22f1c6e503c6 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:51.259145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-11a2adbf-d9cc-411b-94ad-22f1c6e503c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:51.302484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419000af-2e2a-4665-9072-21b90801a90a admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:51.429760 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dabd9f-6847-4276-a5aa-cee03029f268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:51.439900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dabd9f-6847-4276-a5aa-cee03029f268 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:51.440424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dabd9f-6847-4276-a5aa-cee03029f268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:51.440850 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dabd9f-6847-4276-a5aa-cee03029f268 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:51.441835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dabd9f-6847-4276-a5aa-cee03029f268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:51.444163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dabd9f-6847-4276-a5aa-cee03029f268 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:51.471590 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:51.499730 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 855/1691] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:51.503098 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 837/1692] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:34:51 2022] POST /identity/v3/users => generated 300 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:51.512426 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2adbf-d9cc-411b-94ad-22f1c6e503c6 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:51.515597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-11a2adbf-d9cc-411b-94ad-22f1c6e503c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:51.546170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2adbf-d9cc-411b-94ad-22f1c6e503c6 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:51.566118 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215b84c-c5e8-4f41-97a4-b5faeff2a115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:51.571104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215b84c-c5e8-4f41-97a4-b5faeff2a115 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:51.572577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215b84c-c5e8-4f41-97a4-b5faeff2a115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:51.572936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215b84c-c5e8-4f41-97a4-b5faeff2a115 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:51.577182 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215b84c-c5e8-4f41-97a4-b5faeff2a115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:51.580077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215b84c-c5e8-4f41-97a4-b5faeff2a115 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:51.605252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 856/1693] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:51.609487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dabd9f-6847-4276-a5aa-cee03029f268 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:51.627149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dabd9f-6847-4276-a5aa-cee03029f268 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:51.638480 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4fadde-8d47-4cf6-8f81-17a8d54ae7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:51.639854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4fadde-8d47-4cf6-8f81-17a8d54ae7a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:51.645225 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4fadde-8d47-4cf6-8f81-17a8d54ae7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:51.645993 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4fadde-8d47-4cf6-8f81-17a8d54ae7a0 admin admin] PATH_INFO: `/v3/projects/80df8ee25ad54bb0959a5028270cd779/users/4a09416112644eeea6e096d873a7176f/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:51.707111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4fadde-8d47-4cf6-8f81-17a8d54ae7a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80df8ee25ad54bb0959a5028270cd779, user_id=4a09416112644eeea6e096d873a7176f, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:51.710661 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4fadde-8d47-4cf6-8f81-17a8d54ae7a0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:51.745975 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215b84c-c5e8-4f41-97a4-b5faeff2a115 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:51.751175 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 838/1694] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:51 2022] PUT /identity/v3/projects/80df8ee25ad54bb0959a5028270cd779/users/4a09416112644eeea6e096d873a7176f/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:51.759413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215b84c-c5e8-4f41-97a4-b5faeff2a115 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:51.886703 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbe385-310b-4c4b-a301-addbfd7af624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:51.887451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbe385-310b-4c4b-a301-addbfd7af624 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:51.887734 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbe385-310b-4c4b-a301-addbfd7af624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:51.888011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbe385-310b-4c4b-a301-addbfd7af624 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:51.901291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0a5ed-0bdf-4594-b33c-e3000aef0cb8 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:51.901582 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0a5ed-0bdf-4594-b33c-e3000aef0cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:51.901857 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0a5ed-0bdf-4594-b33c-e3000aef0cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:51.920598 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cbe385-310b-4c4b-a301-addbfd7af624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:51.924713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cbe385-310b-4c4b-a301-addbfd7af624 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:51.955945 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 857/1695] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:51.956996 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbe385-310b-4c4b-a301-addbfd7af624 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:51.959579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f2cbe385-310b-4c4b-a301-addbfd7af624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:51.965426 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-3de0a5ed-0bdf-4594-b33c-e3000aef0cb8 None None] MFA Rules not processed for user `16c6975c73954487a1f15649693b1d8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:51.966623 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3de0a5ed-0bdf-4594-b33c-e3000aef0cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:52.071310 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbe385-310b-4c4b-a301-addbfd7af624 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:52.082103 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68c69c-71b8-477d-8d0e-2cad762382db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:52.082782 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68c69c-71b8-477d-8d0e-2cad762382db service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:52.084019 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68c69c-71b8-477d-8d0e-2cad762382db service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:52.084306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68c69c-71b8-477d-8d0e-2cad762382db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:52.113166 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68c69c-71b8-477d-8d0e-2cad762382db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:52.114068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 839/1696] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:52.115051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0a5ed-0bdf-4594-b33c-e3000aef0cb8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:52.119337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68c69c-71b8-477d-8d0e-2cad762382db service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:52.135759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0a5ed-0bdf-4594-b33c-e3000aef0cb8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:52.141743 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 858/1697] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:52.145275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7c376-357d-4fd9-aa22-bebdba98150d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:52.146413 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7c376-357d-4fd9-aa22-bebdba98150d admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:52.147186 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7c376-357d-4fd9-aa22-bebdba98150d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:52.147637 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7c376-357d-4fd9-aa22-bebdba98150d admin admin] PATH_INFO: `/v3/projects/80df8ee25ad54bb0959a5028270cd779/users/4a09416112644eeea6e096d873a7176f/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:52.150104 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7c376-357d-4fd9-aa22-bebdba98150d admin admin] RBAC: Authorizing `identity:create_grant(project_id=80df8ee25ad54bb0959a5028270cd779, user_id=4a09416112644eeea6e096d873a7176f, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:52.153289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7c376-357d-4fd9-aa22-bebdba98150d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:52.174709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 840/1698] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:52 2022] PUT /identity/v3/projects/80df8ee25ad54bb0959a5028270cd779/users/4a09416112644eeea6e096d873a7176f/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:52.177818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbafa2-aa17-4daf-984e-a61dcf57c6c2 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:52.178190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbafa2-aa17-4daf-984e-a61dcf57c6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:52.178543 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbafa2-aa17-4daf-984e-a61dcf57c6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:52.226991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-aabbafa2-aa17-4daf-984e-a61dcf57c6c2 None None] MFA Rules not processed for user `4a09416112644eeea6e096d873a7176f`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:52.228459 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-aabbafa2-aa17-4daf-984e-a61dcf57c6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:52.260268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:52.368638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7c376-357d-4fd9-aa22-bebdba98150d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:52.370662 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e4f7c376-357d-4fd9-aa22-bebdba98150d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:52.402285 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 859/1699] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:52.429116 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbafa2-aa17-4daf-984e-a61dcf57c6c2 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:52.431708 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-aabbafa2-aa17-4daf-984e-a61dcf57c6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:52.598947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbafa2-aa17-4daf-984e-a61dcf57c6c2 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:52.603649 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7c376-357d-4fd9-aa22-bebdba98150d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:52.621773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75c21a-4e08-4a9b-acae-6d9fb8d11fc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:52.622407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e7014-4390-42cc-8535-7d8b80903b16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:52.622853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75c21a-4e08-4a9b-acae-6d9fb8d11fc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:52.622853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75c21a-4e08-4a9b-acae-6d9fb8d11fc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:52.623037 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75c21a-4e08-4a9b-acae-6d9fb8d11fc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:52.623111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e7014-4390-42cc-8535-7d8b80903b16 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:52.623513 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e7014-4390-42cc-8535-7d8b80903b16 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:52.623844 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e7014-4390-42cc-8535-7d8b80903b16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:52.660638 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75c21a-4e08-4a9b-acae-6d9fb8d11fc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:52.662163 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75c21a-4e08-4a9b-acae-6d9fb8d11fc3 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:52.693070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 860/1700] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:52.774879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:52.872785 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75c21a-4e08-4a9b-acae-6d9fb8d11fc3 service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:52.918864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e7014-4390-42cc-8535-7d8b80903b16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:52.923773 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e7014-4390-42cc-8535-7d8b80903b16 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:52.962268 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 841/1701] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:34:52.969735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e7014-4390-42cc-8535-7d8b80903b16 service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:52.973254 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4b6e7014-4390-42cc-8535-7d8b80903b16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:53.055330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75c21a-4e08-4a9b-acae-6d9fb8d11fc3 service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:53.074703 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ec8b0-10ef-48d4-9bad-e11582b7df76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:53.082642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ec8b0-10ef-48d4-9bad-e11582b7df76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:53.082957 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ec8b0-10ef-48d4-9bad-e11582b7df76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:53.083332 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ec8b0-10ef-48d4-9bad-e11582b7df76 admin admin] PATH_INFO: `/v3/users/b75303d11a5f4f9b9742c97faade1613` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:53.086886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ec8b0-10ef-48d4-9bad-e11582b7df76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b75303d11a5f4f9b9742c97faade1613)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:53.092049 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ec8b0-10ef-48d4-9bad-e11582b7df76 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:53.227178 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e7014-4390-42cc-8535-7d8b80903b16 service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:53.233509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30e7a3-9aca-4601-9823-1d96e4c46da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:53.258034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30e7a3-9aca-4601-9823-1d96e4c46da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:53.259621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30e7a3-9aca-4601-9823-1d96e4c46da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:53.259999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30e7a3-9aca-4601-9823-1d96e4c46da6 admin admin] PATH_INFO: `/v3/projects/06aff2402ccc40b19528b26bc2c3b1cd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:53.263559 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30e7a3-9aca-4601-9823-1d96e4c46da6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06aff2402ccc40b19528b26bc2c3b1cd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:53.271210 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30e7a3-9aca-4601-9823-1d96e4c46da6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:53.323072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-016ec8b0-10ef-48d4-9bad-e11582b7df76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b75303d11a5f4f9b9742c97faade1613'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:53.337476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:53.345963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-016ec8b0-10ef-48d4-9bad-e11582b7df76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b75303d11a5f4f9b9742c97faade1613'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:53.364754 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-016ec8b0-10ef-48d4-9bad-e11582b7df76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b75303d11a5f4f9b9742c97faade1613'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:53.366864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-016ec8b0-10ef-48d4-9bad-e11582b7df76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b75303d11a5f4f9b9742c97faade1613'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:53.368626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-016ec8b0-10ef-48d4-9bad-e11582b7df76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b75303d11a5f4f9b9742c97faade1613'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:53.403151 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2c30e7a3-9aca-4601-9823-1d96e4c46da6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06aff2402ccc40b19528b26bc2c3b1cd', 'request_id': 'req-2c30e7a3-9aca-4601-9823-1d96e4c46da6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:53.411411 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2c30e7a3-9aca-4601-9823-1d96e4c46da6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06aff2402ccc40b19528b26bc2c3b1cd', 'request_id': 'req-2c30e7a3-9aca-4601-9823-1d96e4c46da6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:53.420652 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2c30e7a3-9aca-4601-9823-1d96e4c46da6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06aff2402ccc40b19528b26bc2c3b1cd', 'request_id': 'req-2c30e7a3-9aca-4601-9823-1d96e4c46da6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:53.441495 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2c30e7a3-9aca-4601-9823-1d96e4c46da6 admin admin] The token cache is being invalidate because project 06aff2402ccc40b19528b26bc2c3b1cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:34:53.442000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2c30e7a3-9aca-4601-9823-1d96e4c46da6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:53.443010 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2c30e7a3-9aca-4601-9823-1d96e4c46da6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:53.443223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 861/1702] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:34:52 2022] DELETE /identity/v3/users/b75303d11a5f4f9b9742c97faade1613 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:53.445525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 842/1703] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:34:52 2022] DELETE /identity/v3/projects/06aff2402ccc40b19528b26bc2c3b1cd => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:53.453217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d06578-7426-4df5-94a0-c3ff4d6a138d None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:53.453852 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d06578-7426-4df5-94a0-c3ff4d6a138d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:53.457594 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3078d1c-8bd2-4357-b2ca-651c93d5512a None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:53.457932 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3078d1c-8bd2-4357-b2ca-651c93d5512a None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:53.458216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3078d1c-8bd2-4357-b2ca-651c93d5512a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:53.459559 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d06578-7426-4df5-94a0-c3ff4d6a138d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:53.520784 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-c3078d1c-8bd2-4357-b2ca-651c93d5512a None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:53.524141 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f7d06578-7426-4df5-94a0-c3ff4d6a138d None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:53.525371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f7d06578-7426-4df5-94a0-c3ff4d6a138d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:53.525806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-c3078d1c-8bd2-4357-b2ca-651c93d5512a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:53.833062 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 843/1704] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:53.837315 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cae898-9473-45dd-9577-a5ce646d272e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:53.839787 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cae898-9473-45dd-9577-a5ce646d272e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:53.840172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cae898-9473-45dd-9577-a5ce646d272e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:53.870807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 862/1705] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:53.885954 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3078d1c-8bd2-4357-b2ca-651c93d5512a None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:53.887744 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-c3078d1c-8bd2-4357-b2ca-651c93d5512a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:53.902418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e0cae898-9473-45dd-9577-a5ce646d272e None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:53.912061 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e0cae898-9473-45dd-9577-a5ce646d272e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:53.954218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 844/1706] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:53.955571 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cae898-9473-45dd-9577-a5ce646d272e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:53.962633 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:53.986656 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cae898-9473-45dd-9577-a5ce646d272e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:53.998494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3078d1c-8bd2-4357-b2ca-651c93d5512a None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:54.004174 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221e625-ad7b-41bb-98ba-83368d520a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:54.004682 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221e625-ad7b-41bb-98ba-83368d520a1f admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:54.005357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221e625-ad7b-41bb-98ba-83368d520a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:54.005647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221e625-ad7b-41bb-98ba-83368d520a1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:54.006397 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4221e625-ad7b-41bb-98ba-83368d520a1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:34:54.006708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4221e625-ad7b-41bb-98ba-83368d520a1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:54.011634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d82bb4-c52c-4df6-8b1f-5155dfc94c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:54.013344 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d82bb4-c52c-4df6-8b1f-5155dfc94c9d service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:54.013784 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4221e625-ad7b-41bb-98ba-83368d520a1f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:54.013923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d82bb4-c52c-4df6-8b1f-5155dfc94c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:54.014501 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d82bb4-c52c-4df6-8b1f-5155dfc94c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:54.047251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d82bb4-c52c-4df6-8b1f-5155dfc94c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:54.057346 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d82bb4-c52c-4df6-8b1f-5155dfc94c9d service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:54.079713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 845/1707] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:34:54.081770 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221e625-ad7b-41bb-98ba-83368d520a1f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:54.105863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 863/1708] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:34:54.117107 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221e625-ad7b-41bb-98ba-83368d520a1f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:54.135263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d6028-580d-4be9-9b9f-9efdb55f3c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:54.135944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d6028-580d-4be9-9b9f-9efdb55f3c09 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:54.136231 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d6028-580d-4be9-9b9f-9efdb55f3c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:54.136506 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d6028-580d-4be9-9b9f-9efdb55f3c09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:54.147832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d6028-580d-4be9-9b9f-9efdb55f3c09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:54.150128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d6028-580d-4be9-9b9f-9efdb55f3c09 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:54.256352 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 846/1709] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:34:54 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:54.260511 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d82bb4-c52c-4df6-8b1f-5155dfc94c9d service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:54.281205 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d82bb4-c52c-4df6-8b1f-5155dfc94c9d service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:54.579205 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b7e37-70a9-4bf2-8ff9-8a2529601044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:54.582916 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b7e37-70a9-4bf2-8ff9-8a2529601044 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:54.583239 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b7e37-70a9-4bf2-8ff9-8a2529601044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:54.583523 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b7e37-70a9-4bf2-8ff9-8a2529601044 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:54.584254 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b7e37-70a9-4bf2-8ff9-8a2529601044 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:54.586403 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b7e37-70a9-4bf2-8ff9-8a2529601044 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:54.650525 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 864/1710] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:34:54 2022] POST /identity/v3/users => generated 287 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:54.652168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d6028-580d-4be9-9b9f-9efdb55f3c09 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:54.666643 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d6028-580d-4be9-9b9f-9efdb55f3c09 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:54.699995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed414e5a-a362-40d9-829c-640495598b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:54.701212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed414e5a-a362-40d9-829c-640495598b04 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:54.701212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed414e5a-a362-40d9-829c-640495598b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:54.701212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed414e5a-a362-40d9-829c-640495598b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:54.701969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed414e5a-a362-40d9-829c-640495598b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:54.717770 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed414e5a-a362-40d9-829c-640495598b04 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:54.765973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b7e37-70a9-4bf2-8ff9-8a2529601044 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:54.767226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 847/1711] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:54.772312 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:54.792877 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b7e37-70a9-4bf2-8ff9-8a2529601044 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:54.809780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a5354-aa93-4bae-9657-b5f2db7aee7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:54.810497 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a5354-aa93-4bae-9657-b5f2db7aee7b admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:54.810750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a5354-aa93-4bae-9657-b5f2db7aee7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:54.811058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a5354-aa93-4bae-9657-b5f2db7aee7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:54.811798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a5354-aa93-4bae-9657-b5f2db7aee7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:54.814219 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a5354-aa93-4bae-9657-b5f2db7aee7b admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:54.862845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 865/1712] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:54.863993 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed414e5a-a362-40d9-829c-640495598b04 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:54.892316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed414e5a-a362-40d9-829c-640495598b04 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:54.922246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1389b51-efa5-4304-8b17-457033c81927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:54.923490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1389b51-efa5-4304-8b17-457033c81927 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:54.923780 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1389b51-efa5-4304-8b17-457033c81927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:54.924050 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1389b51-efa5-4304-8b17-457033c81927 admin admin] PATH_INFO: `/v3/projects/9ec55138975e48b99ac0f99bf97bb1c0/users/f49f2d3f530f402e96089e89900daaf3/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:54.981070 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1389b51-efa5-4304-8b17-457033c81927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec55138975e48b99ac0f99bf97bb1c0, user_id=f49f2d3f530f402e96089e89900daaf3, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:54.983967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1389b51-efa5-4304-8b17-457033c81927 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:55.024074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 848/1713] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:54 2022] PUT /identity/v3/projects/9ec55138975e48b99ac0f99bf97bb1c0/users/f49f2d3f530f402e96089e89900daaf3/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:55.031418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94051afe-5de3-40b4-bcf0-eb052f82abfd None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:55.031799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94051afe-5de3-40b4-bcf0-eb052f82abfd None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:55.032141 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94051afe-5de3-40b4-bcf0-eb052f82abfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:55.107302 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-94051afe-5de3-40b4-bcf0-eb052f82abfd None None] MFA Rules not processed for user `f49f2d3f530f402e96089e89900daaf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:55.108548 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-94051afe-5de3-40b4-bcf0-eb052f82abfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:55.283407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1389b51-efa5-4304-8b17-457033c81927 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:55.285833 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c1389b51-efa5-4304-8b17-457033c81927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:55.296906 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 866/1714] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:55.298028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94051afe-5de3-40b4-bcf0-eb052f82abfd None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:55.422208 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94051afe-5de3-40b4-bcf0-eb052f82abfd None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:55.423818 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1389b51-efa5-4304-8b17-457033c81927 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:55.429016 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab8420-27db-4f61-94aa-421ed2b9cdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:55.429866 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab8420-27db-4f61-94aa-421ed2b9cdc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:55.430252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab8420-27db-4f61-94aa-421ed2b9cdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:55.431244 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab8420-27db-4f61-94aa-421ed2b9cdc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:55.433226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab8420-27db-4f61-94aa-421ed2b9cdc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:55.435297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268edf20-fdbc-4945-81e9-47c8380fc533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:55.436382 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268edf20-fdbc-4945-81e9-47c8380fc533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:55.436554 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab8420-27db-4f61-94aa-421ed2b9cdc7 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:55.438194 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268edf20-fdbc-4945-81e9-47c8380fc533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:55.438653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268edf20-fdbc-4945-81e9-47c8380fc533 admin admin] PATH_INFO: `/v3/projects/ea74e7cf74ba465ea46af5315466e5b4` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:55.442372 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268edf20-fdbc-4945-81e9-47c8380fc533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea74e7cf74ba465ea46af5315466e5b4)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:55.447524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268edf20-fdbc-4945-81e9-47c8380fc533 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:55.494622 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 867/1715] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:34:55 2022] POST /identity/v3/projects => generated 373 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:55.497110 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab8420-27db-4f61-94aa-421ed2b9cdc7 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:55.576023 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-268edf20-fdbc-4945-81e9-47c8380fc533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea74e7cf74ba465ea46af5315466e5b4', 'request_id': 'req-268edf20-fdbc-4945-81e9-47c8380fc533'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:55.587013 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-268edf20-fdbc-4945-81e9-47c8380fc533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea74e7cf74ba465ea46af5315466e5b4', 'request_id': 'req-268edf20-fdbc-4945-81e9-47c8380fc533'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:55.602778 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-268edf20-fdbc-4945-81e9-47c8380fc533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea74e7cf74ba465ea46af5315466e5b4', 'request_id': 'req-268edf20-fdbc-4945-81e9-47c8380fc533'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:55.664618 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-268edf20-fdbc-4945-81e9-47c8380fc533 admin admin] The token cache is being invalidate because project ea74e7cf74ba465ea46af5315466e5b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:34:55.665094 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-268edf20-fdbc-4945-81e9-47c8380fc533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:55.665800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-268edf20-fdbc-4945-81e9-47c8380fc533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:55.668533 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 849/1716] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:34:55 2022] DELETE /identity/v3/projects/ea74e7cf74ba465ea46af5315466e5b4 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:34:55.672598 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268edf20-fdbc-4945-81e9-47c8380fc533 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:55.674381 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-268edf20-fdbc-4945-81e9-47c8380fc533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:55.701421 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab8420-27db-4f61-94aa-421ed2b9cdc7 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:55.728270 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:55.756856 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:55.903847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268edf20-fdbc-4945-81e9-47c8380fc533 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:55.926780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2007d92-5c69-4e62-a57b-77ef27b5c4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:55.928322 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2007d92-5c69-4e62-a57b-77ef27b5c4f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:55.930155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2007d92-5c69-4e62-a57b-77ef27b5c4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:55.930566 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2007d92-5c69-4e62-a57b-77ef27b5c4f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:55.933015 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2007d92-5c69-4e62-a57b-77ef27b5c4f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:55.935277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a51611-8807-46cd-82a1-8db4fbe01daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:55.938839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2007d92-5c69-4e62-a57b-77ef27b5c4f9 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:55.939027 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a51611-8807-46cd-82a1-8db4fbe01daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:55.947528 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a51611-8807-46cd-82a1-8db4fbe01daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:55.948018 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a51611-8807-46cd-82a1-8db4fbe01daa admin admin] PATH_INFO: `/v3/users/4a09416112644eeea6e096d873a7176f` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:55.959906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a51611-8807-46cd-82a1-8db4fbe01daa admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a09416112644eeea6e096d873a7176f)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:55.963494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: WARNING py.warnings [None req-e9a51611-8807-46cd-82a1-8db4fbe01daa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 20:34:55.963494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: warnings.warn(msg) Jul 14 20:34:55.963494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: Jul 14 20:34:55.980925 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a51611-8807-46cd-82a1-8db4fbe01daa admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:56.014821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 868/1717] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:34:55 2022] POST /identity/v3/users => generated 293 bytes in 519 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:56.015869 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2007d92-5c69-4e62-a57b-77ef27b5c4f9 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:56.028661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b2007d92-5c69-4e62-a57b-77ef27b5c4f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:56.061678 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2007d92-5c69-4e62-a57b-77ef27b5c4f9 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:56.094368 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a8acb-8a27-4be2-8ebe-e3a1f2e2cc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:56.095201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a8acb-8a27-4be2-8ebe-e3a1f2e2cc6a admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:56.095601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a8acb-8a27-4be2-8ebe-e3a1f2e2cc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:56.095985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a8acb-8a27-4be2-8ebe-e3a1f2e2cc6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:56.096948 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a8acb-8a27-4be2-8ebe-e3a1f2e2cc6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:56.098776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a8acb-8a27-4be2-8ebe-e3a1f2e2cc6a admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:56.130474 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 869/1718] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:56.134452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a8acb-8a27-4be2-8ebe-e3a1f2e2cc6a admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:56.144852 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e9a51611-8807-46cd-82a1-8db4fbe01daa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a09416112644eeea6e096d873a7176f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:56.166528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a8acb-8a27-4be2-8ebe-e3a1f2e2cc6a admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:56.170802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e9a51611-8807-46cd-82a1-8db4fbe01daa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a09416112644eeea6e096d873a7176f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:56.172027 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e9a51611-8807-46cd-82a1-8db4fbe01daa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a09416112644eeea6e096d873a7176f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:56.184021 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e9a51611-8807-46cd-82a1-8db4fbe01daa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a09416112644eeea6e096d873a7176f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:56.187310 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121e4bd-5d13-452c-8439-a357d388f70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:56.192011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121e4bd-5d13-452c-8439-a357d388f70a admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:56.192416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121e4bd-5d13-452c-8439-a357d388f70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:56.199359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121e4bd-5d13-452c-8439-a357d388f70a admin admin] PATH_INFO: `/v3/projects/b326d868ab7a43c7a9186dc9d3dbc687/users/9c39999ae2244bd2b2b98c5194f823dc/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:56.246441 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e9a51611-8807-46cd-82a1-8db4fbe01daa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a09416112644eeea6e096d873a7176f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:56.263757 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121e4bd-5d13-452c-8439-a357d388f70a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b326d868ab7a43c7a9186dc9d3dbc687, user_id=9c39999ae2244bd2b2b98c5194f823dc, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:56.268171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121e4bd-5d13-452c-8439-a357d388f70a admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:56.305415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 870/1719] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:56 2022] PUT /identity/v3/projects/b326d868ab7a43c7a9186dc9d3dbc687/users/9c39999ae2244bd2b2b98c5194f823dc/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:56.309742 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121e4bd-5d13-452c-8439-a357d388f70a admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:56.313756 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-c121e4bd-5d13-452c-8439-a357d388f70a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:56.324499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 850/1720] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:34:55 2022] DELETE /identity/v3/users/4a09416112644eeea6e096d873a7176f => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:34:56.641864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121e4bd-5d13-452c-8439-a357d388f70a admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:56.936140 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41898aba-39f2-4fcb-acf3-d8a55d5af7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:56.939674 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41898aba-39f2-4fcb-acf3-d8a55d5af7ba admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:56.940794 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41898aba-39f2-4fcb-acf3-d8a55d5af7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:56.943164 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41898aba-39f2-4fcb-acf3-d8a55d5af7ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:56.944076 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41898aba-39f2-4fcb-acf3-d8a55d5af7ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:56.946619 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41898aba-39f2-4fcb-acf3-d8a55d5af7ba admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:57.006852 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a51611-8807-46cd-82a1-8db4fbe01daa admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:57.011308 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 871/1721] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 702 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:57.028765 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a51611-8807-46cd-82a1-8db4fbe01daa admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:57.032682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcab1ea-5d5d-4d03-b262-8fd66ecc50f3 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:57.033096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcab1ea-5d5d-4d03-b262-8fd66ecc50f3 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:57.033458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcab1ea-5d5d-4d03-b262-8fd66ecc50f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:57.062961 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:57.066568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125f892-864d-40f3-9630-08cc2e6fc7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:57.067491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125f892-864d-40f3-9630-08cc2e6fc7a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:57.067785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125f892-864d-40f3-9630-08cc2e6fc7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:57.068072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125f892-864d-40f3-9630-08cc2e6fc7a7 admin admin] PATH_INFO: `/v3/projects/b326d868ab7a43c7a9186dc9d3dbc687/users/9c39999ae2244bd2b2b98c5194f823dc/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:57.086926 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125f892-864d-40f3-9630-08cc2e6fc7a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b326d868ab7a43c7a9186dc9d3dbc687, user_id=9c39999ae2244bd2b2b98c5194f823dc, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:57.100317 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125f892-864d-40f3-9630-08cc2e6fc7a7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:57.116103 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-cfcab1ea-5d5d-4d03-b262-8fd66ecc50f3 None None] MFA Rules not processed for user `4f75b8b484384554ad59aa20535875e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:57.117350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-cfcab1ea-5d5d-4d03-b262-8fd66ecc50f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:57.169154 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 851/1722] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:57 2022] PUT /identity/v3/projects/b326d868ab7a43c7a9186dc9d3dbc687/users/9c39999ae2244bd2b2b98c5194f823dc/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:34:57.182472 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba026c1b-c371-4460-97dd-88ec312ac955 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:57.182767 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba026c1b-c371-4460-97dd-88ec312ac955 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:57.183309 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba026c1b-c371-4460-97dd-88ec312ac955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:57.255393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 872/1723] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 14 20:34:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:57.345040 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ba026c1b-c371-4460-97dd-88ec312ac955 None None] MFA Rules not processed for user `9c39999ae2244bd2b2b98c5194f823dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:57.348946 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ba026c1b-c371-4460-97dd-88ec312ac955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:57.554691 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcab1ea-5d5d-4d03-b262-8fd66ecc50f3 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:57.557502 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-cfcab1ea-5d5d-4d03-b262-8fd66ecc50f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:57.717845 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 852/1724] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 534 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:57.721111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba026c1b-c371-4460-97dd-88ec312ac955 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:57.728318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ba026c1b-c371-4460-97dd-88ec312ac955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:57.802589 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:57.880862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcab1ea-5d5d-4d03-b262-8fd66ecc50f3 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:57.946770 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba026c1b-c371-4460-97dd-88ec312ac955 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:57.955210 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6439706-ef45-4c12-9eb5-6ca55d698557 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:57.956452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6439706-ef45-4c12-9eb5-6ca55d698557 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:57.956879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6439706-ef45-4c12-9eb5-6ca55d698557 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:57.957285 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6439706-ef45-4c12-9eb5-6ca55d698557 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:57.983672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6439706-ef45-4c12-9eb5-6ca55d698557 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:57.985693 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6439706-ef45-4c12-9eb5-6ca55d698557 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:58.039291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 853/1725] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:58.047809 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997642a5-6292-40ba-bf32-635565915ea8 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:58.048119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997642a5-6292-40ba-bf32-635565915ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:58.048416 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997642a5-6292-40ba-bf32-635565915ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:58.082250 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929cb4b2-a3ee-40c5-a839-71e8fc04be8b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc23233929144a5b7b3a1733da19828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:58.086255 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cb4b2-a3ee-40c5-a839-71e8fc04be8b service swift] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:58.088536 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-997642a5-6292-40ba-bf32-635565915ea8 None None] MFA Rules not processed for user `16c6975c73954487a1f15649693b1d8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:58.089134 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cb4b2-a3ee-40c5-a839-71e8fc04be8b service swift] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:58.089625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cb4b2-a3ee-40c5-a839-71e8fc04be8b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:58.090042 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-997642a5-6292-40ba-bf32-635565915ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:58.100058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:58.226261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 854/1726] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:58.234551 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929cb4b2-a3ee-40c5-a839-71e8fc04be8b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:58.236191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929cb4b2-a3ee-40c5-a839-71e8fc04be8b service swift] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:58.240139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997642a5-6292-40ba-bf32-635565915ea8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:58.277614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997642a5-6292-40ba-bf32-635565915ea8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:58.281736 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 873/1727] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 14 20:34:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 727 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 20:34:58.295108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce54e2-7f30-48ef-a46a-e254f1ca7926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:58.296008 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce54e2-7f30-48ef-a46a-e254f1ca7926 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:58.296258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce54e2-7f30-48ef-a46a-e254f1ca7926 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:58.296540 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce54e2-7f30-48ef-a46a-e254f1ca7926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:58.303865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-bbce54e2-7f30-48ef-a46a-e254f1ca7926 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:58.490818 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce54e2-7f30-48ef-a46a-e254f1ca7926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:58.493908 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce54e2-7f30-48ef-a46a-e254f1ca7926 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:58.523720 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 855/1728] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:34:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:34:58.610366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929cb4b2-a3ee-40c5-a839-71e8fc04be8b service swift] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:58.611962 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-929cb4b2-a3ee-40c5-a839-71e8fc04be8b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:58.841489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929cb4b2-a3ee-40c5-a839-71e8fc04be8b service swift] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:58.864292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f2ff2-851b-4eff-a2e4-e4c8ec3076dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:58.866362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f2ff2-851b-4eff-a2e4-e4c8ec3076dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:58.869543 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f2ff2-851b-4eff-a2e4-e4c8ec3076dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:58.869877 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f2ff2-851b-4eff-a2e4-e4c8ec3076dc admin admin] PATH_INFO: `/v3/projects/80df8ee25ad54bb0959a5028270cd779` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:58.877508 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f2ff2-851b-4eff-a2e4-e4c8ec3076dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=80df8ee25ad54bb0959a5028270cd779)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:58.878136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f2ff2-851b-4eff-a2e4-e4c8ec3076dc admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:58.945694 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d94f2ff2-851b-4eff-a2e4-e4c8ec3076dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80df8ee25ad54bb0959a5028270cd779', 'request_id': 'req-d94f2ff2-851b-4eff-a2e4-e4c8ec3076dc'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:58.961578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d94f2ff2-851b-4eff-a2e4-e4c8ec3076dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80df8ee25ad54bb0959a5028270cd779', 'request_id': 'req-d94f2ff2-851b-4eff-a2e4-e4c8ec3076dc'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:58.969609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d94f2ff2-851b-4eff-a2e4-e4c8ec3076dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80df8ee25ad54bb0959a5028270cd779', 'request_id': 'req-d94f2ff2-851b-4eff-a2e4-e4c8ec3076dc'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:58.982907 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d94f2ff2-851b-4eff-a2e4-e4c8ec3076dc admin admin] The token cache is being invalidate because project 80df8ee25ad54bb0959a5028270cd779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:34:58.983344 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d94f2ff2-851b-4eff-a2e4-e4c8ec3076dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:58.986686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d94f2ff2-851b-4eff-a2e4-e4c8ec3076dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:34:58.991021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 874/1729] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:34:58 2022] DELETE /identity/v3/projects/80df8ee25ad54bb0959a5028270cd779 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:58.993597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177a63a-ad71-4cff-858d-c6f5a475f923 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:58.993891 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177a63a-ad71-4cff-858d-c6f5a475f923 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:58.994167 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177a63a-ad71-4cff-858d-c6f5a475f923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:59.014858 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-b177a63a-ad71-4cff-858d-c6f5a475f923 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:59.016230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b177a63a-ad71-4cff-858d-c6f5a475f923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:59.078321 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:59.089959 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 856/1730] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:59.098819 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea17820-41c9-4583-9ac2-f5aaac3b9b94 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:59.099482 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea17820-41c9-4583-9ac2-f5aaac3b9b94 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:59.099910 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea17820-41c9-4583-9ac2-f5aaac3b9b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:59.121841 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-4ea17820-41c9-4583-9ac2-f5aaac3b9b94 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:59.122998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4ea17820-41c9-4583-9ac2-f5aaac3b9b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:59.137585 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 875/1731] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:59.138869 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177a63a-ad71-4cff-858d-c6f5a475f923 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:59.157582 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177a63a-ad71-4cff-858d-c6f5a475f923 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:59.165288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e957e-2768-4018-ade3-76b7cfadd217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:59.166348 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e957e-2768-4018-ade3-76b7cfadd217 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:59.166835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e957e-2768-4018-ade3-76b7cfadd217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:59.167395 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e957e-2768-4018-ade3-76b7cfadd217 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:59.169841 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e957e-2768-4018-ade3-76b7cfadd217 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:34:59.170678 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e957e-2768-4018-ade3-76b7cfadd217 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:59.175388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e957e-2768-4018-ade3-76b7cfadd217 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:59.202009 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 857/1732] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:34:59.203522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea17820-41c9-4583-9ac2-f5aaac3b9b94 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:59.214948 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea17820-41c9-4583-9ac2-f5aaac3b9b94 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:59.220176 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46555a1-70ed-48ad-8701-7b6011a3af92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:59.220864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46555a1-70ed-48ad-8701-7b6011a3af92 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:59.221146 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46555a1-70ed-48ad-8701-7b6011a3af92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:59.221419 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46555a1-70ed-48ad-8701-7b6011a3af92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:59.222182 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46555a1-70ed-48ad-8701-7b6011a3af92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:59.224184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46555a1-70ed-48ad-8701-7b6011a3af92 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:59.261629 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e957e-2768-4018-ade3-76b7cfadd217 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:59.262054 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 876/1733] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:34:59 2022] POST /identity/v3/projects => generated 409 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:34:59.287969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e957e-2768-4018-ade3-76b7cfadd217 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:59.431818 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e8ddc-3c63-47c5-865b-efb4935248b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:59.432723 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e8ddc-3c63-47c5-865b-efb4935248b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:59.433120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e8ddc-3c63-47c5-865b-efb4935248b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:59.433718 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e8ddc-3c63-47c5-865b-efb4935248b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:59.434557 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e8ddc-3c63-47c5-865b-efb4935248b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:59.436856 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e8ddc-3c63-47c5-865b-efb4935248b3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:59.468694 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 858/1734] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:34:59 2022] POST /identity/v3/users => generated 305 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:34:59.472852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46555a1-70ed-48ad-8701-7b6011a3af92 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:59.490572 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46555a1-70ed-48ad-8701-7b6011a3af92 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:59.514205 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d86880-70c6-4c86-ba7e-3f7a739f372a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:59.514889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d86880-70c6-4c86-ba7e-3f7a739f372a admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:59.515358 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d86880-70c6-4c86-ba7e-3f7a739f372a admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:59.515744 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d86880-70c6-4c86-ba7e-3f7a739f372a admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:59.516479 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d86880-70c6-4c86-ba7e-3f7a739f372a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:59.518543 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d86880-70c6-4c86-ba7e-3f7a739f372a admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:59.530886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:34:59.550545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e8ddc-3c63-47c5-865b-efb4935248b3 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:59.551186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 877/1735] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:34:59.569417 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e8ddc-3c63-47c5-865b-efb4935248b3 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:59.579395 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747e1b9-ae6f-48c2-9589-5f743bf94b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:59.580341 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747e1b9-ae6f-48c2-9589-5f743bf94b92 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:59.581099 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747e1b9-ae6f-48c2-9589-5f743bf94b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:59.581617 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747e1b9-ae6f-48c2-9589-5f743bf94b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:59.585426 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747e1b9-ae6f-48c2-9589-5f743bf94b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:59.591143 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747e1b9-ae6f-48c2-9589-5f743bf94b92 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:59.615778 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 859/1736] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:34:59.620454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d86880-70c6-4c86-ba7e-3f7a739f372a admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:34:59.647999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d86880-70c6-4c86-ba7e-3f7a739f372a admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:34:59.658079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13abcb-596b-4c42-9655-b76f8e7af0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:34:59.659095 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13abcb-596b-4c42-9655-b76f8e7af0d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:59.659392 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13abcb-596b-4c42-9655-b76f8e7af0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:59.659676 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13abcb-596b-4c42-9655-b76f8e7af0d6 admin admin] PATH_INFO: `/v3/projects/dc77ea5b3a3f4cfcb180eac4fae6badb/users/7a939cb235814ff49d29230c0805d6aa/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:59.708004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13abcb-596b-4c42-9655-b76f8e7af0d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc77ea5b3a3f4cfcb180eac4fae6badb, user_id=7a939cb235814ff49d29230c0805d6aa, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:34:59.711194 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13abcb-596b-4c42-9655-b76f8e7af0d6 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:34:59.736242 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 878/1737] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:34:59 2022] PUT /identity/v3/projects/dc77ea5b3a3f4cfcb180eac4fae6badb/users/7a939cb235814ff49d29230c0805d6aa/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:34:59.744734 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd589e1-1291-454c-a204-abab4cf71f48 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:34:59.745021 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd589e1-1291-454c-a204-abab4cf71f48 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:34:59.745454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd589e1-1291-454c-a204-abab4cf71f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:34:59.805593 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-5dd589e1-1291-454c-a204-abab4cf71f48 None None] MFA Rules not processed for user `7a939cb235814ff49d29230c0805d6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:34:59.809117 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5dd589e1-1291-454c-a204-abab4cf71f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:34:59.993243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 860/1738] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:34:59 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:34:59.994889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13abcb-596b-4c42-9655-b76f8e7af0d6 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:00.186805 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13abcb-596b-4c42-9655-b76f8e7af0d6 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:00.201113 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc4a00-f3ea-4bb2-805c-ac4fce55617f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:00.201813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc4a00-f3ea-4bb2-805c-ac4fce55617f admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:00.205648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc4a00-f3ea-4bb2-805c-ac4fce55617f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:00.208315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc4a00-f3ea-4bb2-805c-ac4fce55617f admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:00.211897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc4a00-f3ea-4bb2-805c-ac4fce55617f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:00.221639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc4a00-f3ea-4bb2-805c-ac4fce55617f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:00.323466 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 879/1739] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:34:59 2022] POST /identity/v3/projects => generated 409 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:00.328704 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd589e1-1291-454c-a204-abab4cf71f48 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:00.347435 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd589e1-1291-454c-a204-abab4cf71f48 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:00.495762 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:00.507867 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b17cd-26c9-4a3b-8b91-dda21a141c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:00.508646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b17cd-26c9-4a3b-8b91-dda21a141c95 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:00.509005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b17cd-26c9-4a3b-8b91-dda21a141c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:00.509293 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b17cd-26c9-4a3b-8b91-dda21a141c95 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:00.510102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b17cd-26c9-4a3b-8b91-dda21a141c95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:00.512170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b17cd-26c9-4a3b-8b91-dda21a141c95 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:00.549143 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 861/1740] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:00 2022] POST /identity/v3/users => generated 311 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:00.553009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc4a00-f3ea-4bb2-805c-ac4fce55617f admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:00.568529 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc4a00-f3ea-4bb2-805c-ac4fce55617f admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:00.582361 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf97d63-9796-4924-b0db-50f73217ac92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:00.583625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf97d63-9796-4924-b0db-50f73217ac92 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:00.584203 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf97d63-9796-4924-b0db-50f73217ac92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:00.586072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf97d63-9796-4924-b0db-50f73217ac92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:00.587484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf97d63-9796-4924-b0db-50f73217ac92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:00.596476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf97d63-9796-4924-b0db-50f73217ac92 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:00.621338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 880/1741] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:00.625618 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b17cd-26c9-4a3b-8b91-dda21a141c95 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:00.651556 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b17cd-26c9-4a3b-8b91-dda21a141c95 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:00.668691 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea45e12-abc8-47e2-beb1-ffec197c0f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:00.669783 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea45e12-abc8-47e2-beb1-ffec197c0f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:00.670168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea45e12-abc8-47e2-beb1-ffec197c0f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:00.670710 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea45e12-abc8-47e2-beb1-ffec197c0f1a admin admin] PATH_INFO: `/v3/projects/6cab75615bb04e9c86e3b8260f5992b7/users/89f6ae40173e404aa4e5c5e761fddd3f/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:00.702801 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea45e12-abc8-47e2-beb1-ffec197c0f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cab75615bb04e9c86e3b8260f5992b7, user_id=89f6ae40173e404aa4e5c5e761fddd3f, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:00.707144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea45e12-abc8-47e2-beb1-ffec197c0f1a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:00.791317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf97d63-9796-4924-b0db-50f73217ac92 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:00.798396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 862/1742] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:00 2022] PUT /identity/v3/projects/6cab75615bb04e9c86e3b8260f5992b7/users/89f6ae40173e404aa4e5c5e761fddd3f/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:00.809050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf97d63-9796-4924-b0db-50f73217ac92 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:01.053723 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4a1cc-22d2-4a21-9b09-42feffae95c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:01.056103 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4a1cc-22d2-4a21-9b09-42feffae95c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:01.056702 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4a1cc-22d2-4a21-9b09-42feffae95c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:01.057427 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4a1cc-22d2-4a21-9b09-42feffae95c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:01.058680 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4a1cc-22d2-4a21-9b09-42feffae95c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:01.060815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4a1cc-22d2-4a21-9b09-42feffae95c8 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:01.082563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 881/1743] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:01.085663 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea45e12-abc8-47e2-beb1-ffec197c0f1a admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:01.114470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea45e12-abc8-47e2-beb1-ffec197c0f1a admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:01.130110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5c66e-b321-4fee-8230-22cf11fa6731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:01.131316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5c66e-b321-4fee-8230-22cf11fa6731 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:01.131798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5c66e-b321-4fee-8230-22cf11fa6731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:01.132241 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5c66e-b321-4fee-8230-22cf11fa6731 admin admin] PATH_INFO: `/v3/projects/6cab75615bb04e9c86e3b8260f5992b7/users/89f6ae40173e404aa4e5c5e761fddd3f/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:01.139166 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5c66e-b321-4fee-8230-22cf11fa6731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cab75615bb04e9c86e3b8260f5992b7, user_id=89f6ae40173e404aa4e5c5e761fddd3f, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:01.140854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0095b-0b07-4286-84e6-ba415ee8fed3 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:01.141400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0095b-0b07-4286-84e6-ba415ee8fed3 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:01.142153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0095b-0b07-4286-84e6-ba415ee8fed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:01.143766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5c66e-b321-4fee-8230-22cf11fa6731 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:01.208319 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 863/1744] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:01 2022] PUT /identity/v3/projects/6cab75615bb04e9c86e3b8260f5992b7/users/89f6ae40173e404aa4e5c5e761fddd3f/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:01.208412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-79b0095b-0b07-4286-84e6-ba415ee8fed3 None None] MFA Rules not processed for user `f88257850ee34ad895208ab935b6e643`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:01.208779 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cd0e7-8ac3-4eab-bdad-7d112abb4c68 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:01.209074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cd0e7-8ac3-4eab-bdad-7d112abb4c68 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:01.209358 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cd0e7-8ac3-4eab-bdad-7d112abb4c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:01.211094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-79b0095b-0b07-4286-84e6-ba415ee8fed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:01.302870 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-727cd0e7-8ac3-4eab-bdad-7d112abb4c68 None None] MFA Rules not processed for user `89f6ae40173e404aa4e5c5e761fddd3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:01.304722 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-727cd0e7-8ac3-4eab-bdad-7d112abb4c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:01.439403 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:01.507112 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 882/1745] 10.222.0.62 () {58 vars in 1015 bytes} [Thu Jul 14 20:35:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:01.510663 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 864/1746] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:01 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:01.523765 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0095b-0b07-4286-84e6-ba415ee8fed3 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:01.525633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-79b0095b-0b07-4286-84e6-ba415ee8fed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:01.646179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727cd0e7-8ac3-4eab-bdad-7d112abb4c68 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:01.648327 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-727cd0e7-8ac3-4eab-bdad-7d112abb4c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:01.683830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727cd0e7-8ac3-4eab-bdad-7d112abb4c68 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:01.697913 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e528dc4-d11b-4885-bd5a-72da8acdd218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:01.700424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e528dc4-d11b-4885-bd5a-72da8acdd218 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:01.701035 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e528dc4-d11b-4885-bd5a-72da8acdd218 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:01.701876 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e528dc4-d11b-4885-bd5a-72da8acdd218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:01.718381 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e528dc4-d11b-4885-bd5a-72da8acdd218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:01.722948 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0095b-0b07-4286-84e6-ba415ee8fed3 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:01.727461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e528dc4-d11b-4885-bd5a-72da8acdd218 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:01.734336 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0729837-15c7-4a38-bed8-3093d9664ee3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:01.736543 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0729837-15c7-4a38-bed8-3093d9664ee3 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:01.737861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0729837-15c7-4a38-bed8-3093d9664ee3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:01.738231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0729837-15c7-4a38-bed8-3093d9664ee3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:01.751129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 865/1747] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:35:01.756781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0729837-15c7-4a38-bed8-3093d9664ee3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:01.758338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0729837-15c7-4a38-bed8-3093d9664ee3 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:01.775830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 883/1748] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:01 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:02.589714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e528dc4-d11b-4885-bd5a-72da8acdd218 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:02.591282 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4e528dc4-d11b-4885-bd5a-72da8acdd218 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:02.671532 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e528dc4-d11b-4885-bd5a-72da8acdd218 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:02.689212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec874af-e821-4cbe-9133-7ce40df21f89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:02.692494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec874af-e821-4cbe-9133-7ce40df21f89 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:02.693998 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec874af-e821-4cbe-9133-7ce40df21f89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:02.694335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec874af-e821-4cbe-9133-7ce40df21f89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:03.002106 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec874af-e821-4cbe-9133-7ce40df21f89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:03.004312 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec874af-e821-4cbe-9133-7ce40df21f89 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:03.048406 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 866/1749] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:02 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:35:04.849389 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666b1e1-1a5b-4c05-8b01-c30527fcb319 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:04.849878 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666b1e1-1a5b-4c05-8b01-c30527fcb319 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:04.849967 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666b1e1-1a5b-4c05-8b01-c30527fcb319 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:04.853602 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 884/1750] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 14 20:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:35:05.028285 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c60f0-088e-49c0-8072-83abf490d2f8 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:05.028827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c60f0-088e-49c0-8072-83abf490d2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:05.028827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c60f0-088e-49c0-8072-83abf490d2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:05.108499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-945c60f0-088e-49c0-8072-83abf490d2f8 None None] MFA Rules not processed for user `aa3bb9f1e24a4e9ebd460e97b3736bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:05.109776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-945c60f0-088e-49c0-8072-83abf490d2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:05.157902 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:05.425912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 867/1751] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 14 20:35:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:05.796098 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666b1e1-1a5b-4c05-8b01-c30527fcb319 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:05.798068 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-0666b1e1-1a5b-4c05-8b01-c30527fcb319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:05.927209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666b1e1-1a5b-4c05-8b01-c30527fcb319 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:05.933910 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a5ea1-25bb-4101-b969-fcff27c2d9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:05.935979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a5ea1-25bb-4101-b969-fcff27c2d9a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:05.936273 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a5ea1-25bb-4101-b969-fcff27c2d9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:05.936557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a5ea1-25bb-4101-b969-fcff27c2d9a6 admin admin] PATH_INFO: `/v3/users/68a95688d6c34aaeb5afbaae95fbfa3d` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:05.937847 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a5ea1-25bb-4101-b969-fcff27c2d9a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68a95688d6c34aaeb5afbaae95fbfa3d)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:05.939879 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a5ea1-25bb-4101-b969-fcff27c2d9a6 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:06.019627 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-554a5ea1-25bb-4101-b969-fcff27c2d9a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68a95688d6c34aaeb5afbaae95fbfa3d'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:06.032677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-554a5ea1-25bb-4101-b969-fcff27c2d9a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68a95688d6c34aaeb5afbaae95fbfa3d'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:06.048322 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-554a5ea1-25bb-4101-b969-fcff27c2d9a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68a95688d6c34aaeb5afbaae95fbfa3d'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:06.049536 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-554a5ea1-25bb-4101-b969-fcff27c2d9a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68a95688d6c34aaeb5afbaae95fbfa3d'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:06.051534 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-554a5ea1-25bb-4101-b969-fcff27c2d9a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68a95688d6c34aaeb5afbaae95fbfa3d'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:06.084439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 885/1752] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:05 2022] DELETE /identity/v3/users/68a95688d6c34aaeb5afbaae95fbfa3d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:06.089183 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8a65b-8e50-4094-8100-1ef342ce487e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:06.089782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8a65b-8e50-4094-8100-1ef342ce487e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:06.089782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8a65b-8e50-4094-8100-1ef342ce487e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:06.108968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e4d8a65b-8e50-4094-8100-1ef342ce487e None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:06.110197 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e4d8a65b-8e50-4094-8100-1ef342ce487e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:06.136875 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a5ea1-25bb-4101-b969-fcff27c2d9a6 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:06.139727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-554a5ea1-25bb-4101-b969-fcff27c2d9a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:06.190569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:06.214927 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 868/1753] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:06.245210 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8a65b-8e50-4094-8100-1ef342ce487e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:06.251030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e4d8a65b-8e50-4094-8100-1ef342ce487e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:06.312555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a5ea1-25bb-4101-b969-fcff27c2d9a6 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:06.328501 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f555f9-ab55-40a5-92e8-4a76af7019e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:06.329639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f555f9-ab55-40a5-92e8-4a76af7019e5 service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:06.330241 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f555f9-ab55-40a5-92e8-4a76af7019e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:06.331447 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f555f9-ab55-40a5-92e8-4a76af7019e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:06.334399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d3f555f9-ab55-40a5-92e8-4a76af7019e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:06.351548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8a65b-8e50-4094-8100-1ef342ce487e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:06.355678 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4630b8a-d148-4861-afc6-c98e6bfa8c25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:06.356716 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4630b8a-d148-4861-afc6-c98e6bfa8c25 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:06.357170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4630b8a-d148-4861-afc6-c98e6bfa8c25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:06.357608 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4630b8a-d148-4861-afc6-c98e6bfa8c25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:06.361544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f555f9-ab55-40a5-92e8-4a76af7019e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:06.363405 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f555f9-ab55-40a5-92e8-4a76af7019e5 service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:06.368233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4630b8a-d148-4861-afc6-c98e6bfa8c25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:06.370116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4630b8a-d148-4861-afc6-c98e6bfa8c25 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:06.389130 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 869/1754] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:06.391402 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 886/1755] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:06.663280 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5dc9f9-4feb-4a2d-8158-043d3da6174e None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:06.663765 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5dc9f9-4feb-4a2d-8158-043d3da6174e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:06.664058 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5dc9f9-4feb-4a2d-8158-043d3da6174e None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:06.665608 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 870/1756] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 14 20:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:35:06.690707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4336c7-ef24-46ba-828c-6868b6f662a5 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:06.691483 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4336c7-ef24-46ba-828c-6868b6f662a5 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:06.692209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4336c7-ef24-46ba-828c-6868b6f662a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:06.714320 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-1b4336c7-ef24-46ba-828c-6868b6f662a5 None None] MFA Rules not processed for user `aa3bb9f1e24a4e9ebd460e97b3736bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:06.715737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-1b4336c7-ef24-46ba-828c-6868b6f662a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:06.731496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 887/1757] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 14 20:35:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:06.748040 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5dc9f9-4feb-4a2d-8158-043d3da6174e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:06.793233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5dc9f9-4feb-4a2d-8158-043d3da6174e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:06.801661 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16004b3-8012-49db-abdd-5e691a0e1c6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:06.802331 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16004b3-8012-49db-abdd-5e691a0e1c6f service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:06.802616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16004b3-8012-49db-abdd-5e691a0e1c6f service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:06.802888 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16004b3-8012-49db-abdd-5e691a0e1c6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:06.812589 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16004b3-8012-49db-abdd-5e691a0e1c6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:06.814219 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16004b3-8012-49db-abdd-5e691a0e1c6f service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:06.841116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 871/1758] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:06.979111 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4336c7-ef24-46ba-828c-6868b6f662a5 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:06.980587 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-1b4336c7-ef24-46ba-828c-6868b6f662a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:06.995854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4336c7-ef24-46ba-828c-6868b6f662a5 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:07.000954 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41958fea-cdbe-4986-8cd1-7df511bd682e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:07.001777 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41958fea-cdbe-4986-8cd1-7df511bd682e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:07.002062 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41958fea-cdbe-4986-8cd1-7df511bd682e admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:07.002371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41958fea-cdbe-4986-8cd1-7df511bd682e admin admin] PATH_INFO: `/v3/projects/648bdbfd9b0c45349c25e9efae8e259e` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:07.003740 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41958fea-cdbe-4986-8cd1-7df511bd682e admin admin] RBAC: Authorizing `identity:delete_project(project_id=648bdbfd9b0c45349c25e9efae8e259e)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:07.005927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41958fea-cdbe-4986-8cd1-7df511bd682e admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:07.057072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-41958fea-cdbe-4986-8cd1-7df511bd682e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '648bdbfd9b0c45349c25e9efae8e259e', 'request_id': 'req-41958fea-cdbe-4986-8cd1-7df511bd682e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:07.065397 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-41958fea-cdbe-4986-8cd1-7df511bd682e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '648bdbfd9b0c45349c25e9efae8e259e', 'request_id': 'req-41958fea-cdbe-4986-8cd1-7df511bd682e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:07.072415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-41958fea-cdbe-4986-8cd1-7df511bd682e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '648bdbfd9b0c45349c25e9efae8e259e', 'request_id': 'req-41958fea-cdbe-4986-8cd1-7df511bd682e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:07.083994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-41958fea-cdbe-4986-8cd1-7df511bd682e admin admin] The token cache is being invalidate because project 648bdbfd9b0c45349c25e9efae8e259e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:07.084287 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-41958fea-cdbe-4986-8cd1-7df511bd682e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:07.084975 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-41958fea-cdbe-4986-8cd1-7df511bd682e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:07.086577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 888/1759] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:06 2022] DELETE /identity/v3/projects/648bdbfd9b0c45349c25e9efae8e259e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:07.090611 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16004b3-8012-49db-abdd-5e691a0e1c6f service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:07.102101 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16004b3-8012-49db-abdd-5e691a0e1c6f service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:07.142262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:07.170357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1071a0-7e16-46bd-84e3-be4901152133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:07.171443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1071a0-7e16-46bd-84e3-be4901152133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:07.171729 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1071a0-7e16-46bd-84e3-be4901152133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:07.172028 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1071a0-7e16-46bd-84e3-be4901152133 admin admin] PATH_INFO: `/v3/users/2c33c3cb623e4ff1b12cb567b89ecd47` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:07.173425 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1071a0-7e16-46bd-84e3-be4901152133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c33c3cb623e4ff1b12cb567b89ecd47)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:07.175750 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1071a0-7e16-46bd-84e3-be4901152133 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:07.231522 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3b1071a0-7e16-46bd-84e3-be4901152133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c33c3cb623e4ff1b12cb567b89ecd47'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:07.241758 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3b1071a0-7e16-46bd-84e3-be4901152133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c33c3cb623e4ff1b12cb567b89ecd47'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:07.242502 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3b1071a0-7e16-46bd-84e3-be4901152133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c33c3cb623e4ff1b12cb567b89ecd47'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:07.243228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3b1071a0-7e16-46bd-84e3-be4901152133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c33c3cb623e4ff1b12cb567b89ecd47'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:07.257229 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3b1071a0-7e16-46bd-84e3-be4901152133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c33c3cb623e4ff1b12cb567b89ecd47'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:07.270964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 872/1760] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:07 2022] DELETE /identity/v3/users/2c33c3cb623e4ff1b12cb567b89ecd47 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:07.561822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41958fea-cdbe-4986-8cd1-7df511bd682e admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:07.563455 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-41958fea-cdbe-4986-8cd1-7df511bd682e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:07.579771 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41958fea-cdbe-4986-8cd1-7df511bd682e admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:07.647452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc0ea2-8b98-46f6-b398-f48ac34f2f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:07.648333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc0ea2-8b98-46f6-b398-f48ac34f2f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:07.648619 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc0ea2-8b98-46f6-b398-f48ac34f2f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:07.648899 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc0ea2-8b98-46f6-b398-f48ac34f2f5b admin admin] PATH_INFO: `/v3/projects/4e0754f3cd9543dbafacf74059719645` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:07.650099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc0ea2-8b98-46f6-b398-f48ac34f2f5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e0754f3cd9543dbafacf74059719645)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:07.652035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc0ea2-8b98-46f6-b398-f48ac34f2f5b admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:07.673157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:07.700447 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-08bc0ea2-8b98-46f6-b398-f48ac34f2f5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e0754f3cd9543dbafacf74059719645', 'request_id': 'req-08bc0ea2-8b98-46f6-b398-f48ac34f2f5b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:07.708112 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-08bc0ea2-8b98-46f6-b398-f48ac34f2f5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e0754f3cd9543dbafacf74059719645', 'request_id': 'req-08bc0ea2-8b98-46f6-b398-f48ac34f2f5b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:07.714777 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-08bc0ea2-8b98-46f6-b398-f48ac34f2f5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e0754f3cd9543dbafacf74059719645', 'request_id': 'req-08bc0ea2-8b98-46f6-b398-f48ac34f2f5b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:07.725348 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-08bc0ea2-8b98-46f6-b398-f48ac34f2f5b admin admin] The token cache is being invalidate because project 4e0754f3cd9543dbafacf74059719645 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:07.725636 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-08bc0ea2-8b98-46f6-b398-f48ac34f2f5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:07.726280 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-08bc0ea2-8b98-46f6-b398-f48ac34f2f5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:07.727868 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 889/1761] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:07 2022] DELETE /identity/v3/projects/4e0754f3cd9543dbafacf74059719645 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:07.732730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366c9ef-8995-4747-87e6-6eaf7108fd2c None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:07.733106 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366c9ef-8995-4747-87e6-6eaf7108fd2c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:07.733456 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366c9ef-8995-4747-87e6-6eaf7108fd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:07.751904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-5366c9ef-8995-4747-87e6-6eaf7108fd2c None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:07.753693 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5366c9ef-8995-4747-87e6-6eaf7108fd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:07.812526 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 873/1762] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:07.823047 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671a93e-6bc9-4ddb-a5ac-bba49c9554e7 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:07.823339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671a93e-6bc9-4ddb-a5ac-bba49c9554e7 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:07.823619 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671a93e-6bc9-4ddb-a5ac-bba49c9554e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:07.846650 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-0671a93e-6bc9-4ddb-a5ac-bba49c9554e7 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:07.848031 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-0671a93e-6bc9-4ddb-a5ac-bba49c9554e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:07.855772 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 890/1763] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:07.860034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366c9ef-8995-4747-87e6-6eaf7108fd2c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:07.870376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366c9ef-8995-4747-87e6-6eaf7108fd2c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:07.875268 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b89db-71f0-4174-b828-27cfc568e0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:07.875937 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b89db-71f0-4174-b828-27cfc568e0be admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:07.876433 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b89db-71f0-4174-b828-27cfc568e0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:07.876555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b89db-71f0-4174-b828-27cfc568e0be admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:07.882960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b89db-71f0-4174-b828-27cfc568e0be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:35:07.882960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b89db-71f0-4174-b828-27cfc568e0be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:07.882960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b89db-71f0-4174-b828-27cfc568e0be admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:07.896111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 874/1764] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:35:07.902272 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671a93e-6bc9-4ddb-a5ac-bba49c9554e7 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:07.913449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671a93e-6bc9-4ddb-a5ac-bba49c9554e7 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:07.919347 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1bb9c-73f6-49bc-9cb1-7ea6b2fb4a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:07.920200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1bb9c-73f6-49bc-9cb1-7ea6b2fb4a8f admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:07.920561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1bb9c-73f6-49bc-9cb1-7ea6b2fb4a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:07.920912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1bb9c-73f6-49bc-9cb1-7ea6b2fb4a8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:07.921712 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1bb9c-73f6-49bc-9cb1-7ea6b2fb4a8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:07.924148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1bb9c-73f6-49bc-9cb1-7ea6b2fb4a8f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:07.960267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 891/1765] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:07 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:35:07.964592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b89db-71f0-4174-b828-27cfc568e0be admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:07.979012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b89db-71f0-4174-b828-27cfc568e0be admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:08.038562 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:08.076034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21731e4c-484d-402e-8514-5d3782ff3ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:08.076939 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21731e4c-484d-402e-8514-5d3782ff3ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:08.077308 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21731e4c-484d-402e-8514-5d3782ff3ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:08.077834 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21731e4c-484d-402e-8514-5d3782ff3ea6 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:08.078716 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21731e4c-484d-402e-8514-5d3782ff3ea6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:08.081028 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21731e4c-484d-402e-8514-5d3782ff3ea6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:08.121996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 875/1766] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:07 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:08.123927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1bb9c-73f6-49bc-9cb1-7ea6b2fb4a8f admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:08.136991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1bb9c-73f6-49bc-9cb1-7ea6b2fb4a8f admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:08.142274 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5bc71-651b-4fc0-a867-f7bb4dd98192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:08.142981 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5bc71-651b-4fc0-a867-f7bb4dd98192 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:08.143316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5bc71-651b-4fc0-a867-f7bb4dd98192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:08.143594 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5bc71-651b-4fc0-a867-f7bb4dd98192 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:08.144482 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5bc71-651b-4fc0-a867-f7bb4dd98192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:08.146090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5bc71-651b-4fc0-a867-f7bb4dd98192 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:08.165308 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 892/1767] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:08.165510 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21731e4c-484d-402e-8514-5d3782ff3ea6 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:08.180896 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21731e4c-484d-402e-8514-5d3782ff3ea6 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:08.185230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052adb2-1093-4f54-836b-f805aa97655c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:08.185908 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052adb2-1093-4f54-836b-f805aa97655c admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:08.186187 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052adb2-1093-4f54-836b-f805aa97655c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:08.186476 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052adb2-1093-4f54-836b-f805aa97655c admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:08.187234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052adb2-1093-4f54-836b-f805aa97655c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:08.188841 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052adb2-1093-4f54-836b-f805aa97655c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:08.199575 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 876/1768] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:08.204384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5bc71-651b-4fc0-a867-f7bb4dd98192 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:08.216093 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5bc71-651b-4fc0-a867-f7bb4dd98192 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:08.222215 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27473b-3308-43d6-b1a4-33270d38999d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:08.223393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27473b-3308-43d6-b1a4-33270d38999d admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:08.223755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27473b-3308-43d6-b1a4-33270d38999d admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:08.224119 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27473b-3308-43d6-b1a4-33270d38999d admin admin] PATH_INFO: `/v3/projects/c97bbbd7146d48648d9cc65849bc2ed7/users/ad9b7fda1f4b43de97e85483a5e1d958/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:08.247927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27473b-3308-43d6-b1a4-33270d38999d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c97bbbd7146d48648d9cc65849bc2ed7, user_id=ad9b7fda1f4b43de97e85483a5e1d958, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:08.250847 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27473b-3308-43d6-b1a4-33270d38999d admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:08.273783 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 893/1769] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:08 2022] PUT /identity/v3/projects/c97bbbd7146d48648d9cc65849bc2ed7/users/ad9b7fda1f4b43de97e85483a5e1d958/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:08.288214 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dc747-2929-41ee-ac4b-5ab8ddd0d93b None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:08.288693 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dc747-2929-41ee-ac4b-5ab8ddd0d93b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:08.288985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dc747-2929-41ee-ac4b-5ab8ddd0d93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:08.329866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-997dc747-2929-41ee-ac4b-5ab8ddd0d93b None None] MFA Rules not processed for user `ad9b7fda1f4b43de97e85483a5e1d958`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:08.331197 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-997dc747-2929-41ee-ac4b-5ab8ddd0d93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:08.463993 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 877/1770] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:35:08.468710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27473b-3308-43d6-b1a4-33270d38999d admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:08.550510 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27473b-3308-43d6-b1a4-33270d38999d admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:08.555458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac084f3-86ee-4a9f-969c-ebb419c4e48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:08.556138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac084f3-86ee-4a9f-969c-ebb419c4e48f admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:08.556423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac084f3-86ee-4a9f-969c-ebb419c4e48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:08.556705 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac084f3-86ee-4a9f-969c-ebb419c4e48f admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:08.557426 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac084f3-86ee-4a9f-969c-ebb419c4e48f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:08.559954 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac084f3-86ee-4a9f-969c-ebb419c4e48f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:08.602472 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 894/1771] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:08 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:08.606223 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dc747-2929-41ee-ac4b-5ab8ddd0d93b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:08.617469 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dc747-2929-41ee-ac4b-5ab8ddd0d93b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:08.705522 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df880d44-4b6f-43b3-b067-c79acebac09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:08.706728 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df880d44-4b6f-43b3-b067-c79acebac09d admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:08.707039 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df880d44-4b6f-43b3-b067-c79acebac09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:08.707316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df880d44-4b6f-43b3-b067-c79acebac09d admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:08.708016 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df880d44-4b6f-43b3-b067-c79acebac09d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:08.709972 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df880d44-4b6f-43b3-b067-c79acebac09d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:08.733616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 878/1772] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:08 2022] POST /identity/v3/users => generated 299 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:08.738156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac084f3-86ee-4a9f-969c-ebb419c4e48f admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:08.748589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac084f3-86ee-4a9f-969c-ebb419c4e48f admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:08.753201 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9aab8-444d-444e-a641-057966ffca35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:08.753884 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9aab8-444d-444e-a641-057966ffca35 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:08.754170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9aab8-444d-444e-a641-057966ffca35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:08.754456 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9aab8-444d-444e-a641-057966ffca35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:08.755277 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9aab8-444d-444e-a641-057966ffca35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:08.756864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9aab8-444d-444e-a641-057966ffca35 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:08.767567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 895/1773] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:08.772956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df880d44-4b6f-43b3-b067-c79acebac09d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:08.785384 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df880d44-4b6f-43b3-b067-c79acebac09d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:08.791931 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154dac8-f48e-4da7-aa67-56128f25ae4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:08.792897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154dac8-f48e-4da7-aa67-56128f25ae4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:08.793189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154dac8-f48e-4da7-aa67-56128f25ae4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:08.793469 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154dac8-f48e-4da7-aa67-56128f25ae4a admin admin] PATH_INFO: `/v3/projects/d72d40be54f54b4bba7f2c2bc2f08afb/users/0361723824374976b7b2392c7311525c/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:08.813620 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154dac8-f48e-4da7-aa67-56128f25ae4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d72d40be54f54b4bba7f2c2bc2f08afb, user_id=0361723824374976b7b2392c7311525c, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:08.817371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154dac8-f48e-4da7-aa67-56128f25ae4a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:08.831581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 879/1774] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:08 2022] PUT /identity/v3/projects/d72d40be54f54b4bba7f2c2bc2f08afb/users/0361723824374976b7b2392c7311525c/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:08.836204 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9aab8-444d-444e-a641-057966ffca35 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:08.847318 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9aab8-444d-444e-a641-057966ffca35 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:08.869846 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:08.940804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ec955-c80a-4538-97c1-86fd493f900f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:08.941525 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ec955-c80a-4538-97c1-86fd493f900f admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:08.942036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ec955-c80a-4538-97c1-86fd493f900f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:08.942245 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ec955-c80a-4538-97c1-86fd493f900f admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:08.943173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ec955-c80a-4538-97c1-86fd493f900f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:08.944820 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ec955-c80a-4538-97c1-86fd493f900f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:08.957058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 896/1775] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:08.960558 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154dac8-f48e-4da7-aa67-56128f25ae4a admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:08.978659 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154dac8-f48e-4da7-aa67-56128f25ae4a admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:08.984404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb84e6-4840-4467-aad1-c4577ceb0297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:08.985403 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb84e6-4840-4467-aad1-c4577ceb0297 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:08.997100 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb84e6-4840-4467-aad1-c4577ceb0297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:08.997425 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb84e6-4840-4467-aad1-c4577ceb0297 admin admin] PATH_INFO: `/v3/projects/d72d40be54f54b4bba7f2c2bc2f08afb/users/0361723824374976b7b2392c7311525c/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:09.004875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb84e6-4840-4467-aad1-c4577ceb0297 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d72d40be54f54b4bba7f2c2bc2f08afb, user_id=0361723824374976b7b2392c7311525c, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:09.008175 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb84e6-4840-4467-aad1-c4577ceb0297 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:09.033377 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 880/1776] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:08 2022] PUT /identity/v3/projects/d72d40be54f54b4bba7f2c2bc2f08afb/users/0361723824374976b7b2392c7311525c/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:35:09.044107 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb05af5-ef60-4743-b2cb-e1f88bff122a None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:09.044547 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb05af5-ef60-4743-b2cb-e1f88bff122a None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:09.044762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb05af5-ef60-4743-b2cb-e1f88bff122a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:09.076060 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-ceb05af5-ef60-4743-b2cb-e1f88bff122a None None] MFA Rules not processed for user `0361723824374976b7b2392c7311525c`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:09.077362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ceb05af5-ef60-4743-b2cb-e1f88bff122a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:09.152162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 897/1777] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:09.163827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb84e6-4840-4467-aad1-c4577ceb0297 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:09.165772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-69bb84e6-4840-4467-aad1-c4577ceb0297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:09.180936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:09.252301 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb84e6-4840-4467-aad1-c4577ceb0297 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:09.258988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3007ce4-465c-446f-88b4-f68b63bb7ad2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:09.259203 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3007ce4-465c-446f-88b4-f68b63bb7ad2 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:09.259431 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3007ce4-465c-446f-88b4-f68b63bb7ad2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:09.259884 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3007ce4-465c-446f-88b4-f68b63bb7ad2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:09.270457 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3007ce4-465c-446f-88b4-f68b63bb7ad2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:09.272557 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3007ce4-465c-446f-88b4-f68b63bb7ad2 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:09.286017 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 881/1778] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:09 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:09.313947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb05af5-ef60-4743-b2cb-e1f88bff122a None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:09.399962 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb05af5-ef60-4743-b2cb-e1f88bff122a None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:09.404927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c594bf-f42b-40c0-82ae-8bc557b5215c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:09.406114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c594bf-f42b-40c0-82ae-8bc557b5215c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:09.406550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c594bf-f42b-40c0-82ae-8bc557b5215c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:09.407298 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c594bf-f42b-40c0-82ae-8bc557b5215c admin admin] PATH_INFO: `/v3/users/ad9b7fda1f4b43de97e85483a5e1d958` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:09.409426 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c594bf-f42b-40c0-82ae-8bc557b5215c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad9b7fda1f4b43de97e85483a5e1d958)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:09.411890 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c594bf-f42b-40c0-82ae-8bc557b5215c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:09.480026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:09.502591 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-33c594bf-f42b-40c0-82ae-8bc557b5215c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad9b7fda1f4b43de97e85483a5e1d958'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:09.519284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-33c594bf-f42b-40c0-82ae-8bc557b5215c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad9b7fda1f4b43de97e85483a5e1d958'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:09.536425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-33c594bf-f42b-40c0-82ae-8bc557b5215c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad9b7fda1f4b43de97e85483a5e1d958'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:09.537590 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-33c594bf-f42b-40c0-82ae-8bc557b5215c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad9b7fda1f4b43de97e85483a5e1d958'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:09.538421 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-33c594bf-f42b-40c0-82ae-8bc557b5215c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad9b7fda1f4b43de97e85483a5e1d958'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:09.576478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 898/1779] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:09 2022] DELETE /identity/v3/users/ad9b7fda1f4b43de97e85483a5e1d958 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:09.581289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0fe0d-cd87-4d89-8ee4-fc29d6b5fe05 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:09.581708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0fe0d-cd87-4d89-8ee4-fc29d6b5fe05 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:09.582798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0fe0d-cd87-4d89-8ee4-fc29d6b5fe05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:09.604435 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8da0fe0d-cd87-4d89-8ee4-fc29d6b5fe05 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:09.606836 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8da0fe0d-cd87-4d89-8ee4-fc29d6b5fe05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:09.700327 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 882/1780] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:09.710177 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c594bf-f42b-40c0-82ae-8bc557b5215c admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:09.711688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-33c594bf-f42b-40c0-82ae-8bc557b5215c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:09.801238 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c594bf-f42b-40c0-82ae-8bc557b5215c admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:09.806233 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad73b2-c7dd-4d79-a6d7-698604e2a0da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:09.808548 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad73b2-c7dd-4d79-a6d7-698604e2a0da service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:09.809097 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad73b2-c7dd-4d79-a6d7-698604e2a0da service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:09.810306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad73b2-c7dd-4d79-a6d7-698604e2a0da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:09.823677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad73b2-c7dd-4d79-a6d7-698604e2a0da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:09.826005 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad73b2-c7dd-4d79-a6d7-698604e2a0da service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:09.839604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 899/1781] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:10.196341 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0fe0d-cd87-4d89-8ee4-fc29d6b5fe05 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:10.198045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8da0fe0d-cd87-4d89-8ee4-fc29d6b5fe05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:10.227912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0fe0d-cd87-4d89-8ee4-fc29d6b5fe05 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:10.233244 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1fc92-8e19-43d3-a3ec-e46f11cb9091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:10.234123 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1fc92-8e19-43d3-a3ec-e46f11cb9091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:10.234410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1fc92-8e19-43d3-a3ec-e46f11cb9091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:10.234685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1fc92-8e19-43d3-a3ec-e46f11cb9091 admin admin] PATH_INFO: `/v3/projects/c97bbbd7146d48648d9cc65849bc2ed7` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:10.236070 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1fc92-8e19-43d3-a3ec-e46f11cb9091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c97bbbd7146d48648d9cc65849bc2ed7)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:10.238091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1fc92-8e19-43d3-a3ec-e46f11cb9091 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:10.297873 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a2a1fc92-8e19-43d3-a3ec-e46f11cb9091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c97bbbd7146d48648d9cc65849bc2ed7', 'request_id': 'req-a2a1fc92-8e19-43d3-a3ec-e46f11cb9091'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:10.304358 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a2a1fc92-8e19-43d3-a3ec-e46f11cb9091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c97bbbd7146d48648d9cc65849bc2ed7', 'request_id': 'req-a2a1fc92-8e19-43d3-a3ec-e46f11cb9091'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:10.309814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a2a1fc92-8e19-43d3-a3ec-e46f11cb9091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c97bbbd7146d48648d9cc65849bc2ed7', 'request_id': 'req-a2a1fc92-8e19-43d3-a3ec-e46f11cb9091'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:10.322783 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a2a1fc92-8e19-43d3-a3ec-e46f11cb9091 admin admin] The token cache is being invalidate because project c97bbbd7146d48648d9cc65849bc2ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:10.323173 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a2a1fc92-8e19-43d3-a3ec-e46f11cb9091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:10.323792 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a2a1fc92-8e19-43d3-a3ec-e46f11cb9091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:10.325350 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 883/1782] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:10 2022] DELETE /identity/v3/projects/c97bbbd7146d48648d9cc65849bc2ed7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:10.329549 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad73b2-c7dd-4d79-a6d7-698604e2a0da service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:10.339768 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad73b2-c7dd-4d79-a6d7-698604e2a0da service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:10.416841 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f47f0-1d34-47d0-8f42-51d444fb2736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:10.418057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f47f0-1d34-47d0-8f42-51d444fb2736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:10.418450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f47f0-1d34-47d0-8f42-51d444fb2736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:10.418771 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f47f0-1d34-47d0-8f42-51d444fb2736 admin admin] PATH_INFO: `/v3/users/0361723824374976b7b2392c7311525c` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:10.420210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f47f0-1d34-47d0-8f42-51d444fb2736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0361723824374976b7b2392c7311525c)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:10.422920 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f47f0-1d34-47d0-8f42-51d444fb2736 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:10.501543 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3b7f47f0-1d34-47d0-8f42-51d444fb2736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0361723824374976b7b2392c7311525c'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:10.510055 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3b7f47f0-1d34-47d0-8f42-51d444fb2736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0361723824374976b7b2392c7311525c'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:10.527526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3b7f47f0-1d34-47d0-8f42-51d444fb2736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0361723824374976b7b2392c7311525c'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:10.528488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3b7f47f0-1d34-47d0-8f42-51d444fb2736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0361723824374976b7b2392c7311525c'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:10.529213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3b7f47f0-1d34-47d0-8f42-51d444fb2736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0361723824374976b7b2392c7311525c'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:10.530658 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:10.566217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 900/1783] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:10 2022] DELETE /identity/v3/users/0361723824374976b7b2392c7311525c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:11.008779 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1fc92-8e19-43d3-a3ec-e46f11cb9091 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:11.010274 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a2a1fc92-8e19-43d3-a3ec-e46f11cb9091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:11.028889 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1fc92-8e19-43d3-a3ec-e46f11cb9091 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:11.067551 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:11.117017 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16aed34-efee-4687-bbac-1e7496253f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:11.118334 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16aed34-efee-4687-bbac-1e7496253f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:11.118697 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16aed34-efee-4687-bbac-1e7496253f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:11.119073 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16aed34-efee-4687-bbac-1e7496253f68 admin admin] PATH_INFO: `/v3/projects/d72d40be54f54b4bba7f2c2bc2f08afb` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:11.120557 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16aed34-efee-4687-bbac-1e7496253f68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d72d40be54f54b4bba7f2c2bc2f08afb)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:11.122614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16aed34-efee-4687-bbac-1e7496253f68 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:11.219635 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d16aed34-efee-4687-bbac-1e7496253f68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd72d40be54f54b4bba7f2c2bc2f08afb', 'request_id': 'req-d16aed34-efee-4687-bbac-1e7496253f68'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:11.231900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d16aed34-efee-4687-bbac-1e7496253f68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd72d40be54f54b4bba7f2c2bc2f08afb', 'request_id': 'req-d16aed34-efee-4687-bbac-1e7496253f68'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:11.241343 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d16aed34-efee-4687-bbac-1e7496253f68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd72d40be54f54b4bba7f2c2bc2f08afb', 'request_id': 'req-d16aed34-efee-4687-bbac-1e7496253f68'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:11.257936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d16aed34-efee-4687-bbac-1e7496253f68 admin admin] The token cache is being invalidate because project d72d40be54f54b4bba7f2c2bc2f08afb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:11.258354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d16aed34-efee-4687-bbac-1e7496253f68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:11.259079 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d16aed34-efee-4687-bbac-1e7496253f68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:11.260718 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 884/1784] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:11 2022] DELETE /identity/v3/projects/d72d40be54f54b4bba7f2c2bc2f08afb => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:11.265726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709ea10-0d8c-4b3f-9a0a-beba0d72dfaf None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:11.266630 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709ea10-0d8c-4b3f-9a0a-beba0d72dfaf None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:11.267088 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709ea10-0d8c-4b3f-9a0a-beba0d72dfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:11.291137 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-c709ea10-0d8c-4b3f-9a0a-beba0d72dfaf None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:11.292559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-c709ea10-0d8c-4b3f-9a0a-beba0d72dfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:11.375948 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 901/1785] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:11.388311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598c089-3a23-4904-be4e-c5e40a70c8a0 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:11.388311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598c089-3a23-4904-be4e-c5e40a70c8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:11.388311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598c089-3a23-4904-be4e-c5e40a70c8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:11.411481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-5598c089-3a23-4904-be4e-c5e40a70c8a0 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:11.412638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5598c089-3a23-4904-be4e-c5e40a70c8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:11.422159 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 885/1786] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:11.426425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709ea10-0d8c-4b3f-9a0a-beba0d72dfaf None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:11.449542 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709ea10-0d8c-4b3f-9a0a-beba0d72dfaf None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:11.456644 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18025b8-89c1-42d5-b54e-726604a242a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:11.457372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18025b8-89c1-42d5-b54e-726604a242a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:11.457660 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18025b8-89c1-42d5-b54e-726604a242a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:11.457934 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18025b8-89c1-42d5-b54e-726604a242a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:11.458794 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18025b8-89c1-42d5-b54e-726604a242a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:35:11.459135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18025b8-89c1-42d5-b54e-726604a242a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:11.460761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18025b8-89c1-42d5-b54e-726604a242a8 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:11.474856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 902/1787] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:35:11.479654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598c089-3a23-4904-be4e-c5e40a70c8a0 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:11.493196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598c089-3a23-4904-be4e-c5e40a70c8a0 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:11.497925 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967aadf8-8755-47de-8f16-7abf64624c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:11.498598 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967aadf8-8755-47de-8f16-7abf64624c0c admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:11.499340 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967aadf8-8755-47de-8f16-7abf64624c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:11.499624 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967aadf8-8755-47de-8f16-7abf64624c0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:11.500325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967aadf8-8755-47de-8f16-7abf64624c0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:11.502272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967aadf8-8755-47de-8f16-7abf64624c0c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:11.543686 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 886/1788] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:11 2022] POST /identity/v3/projects => generated 381 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:11.544648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18025b8-89c1-42d5-b54e-726604a242a8 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:11.555857 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18025b8-89c1-42d5-b54e-726604a242a8 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:11.644586 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f66762-a7fe-46ab-8521-3677250f43cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:11.645739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f66762-a7fe-46ab-8521-3677250f43cb admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:11.646174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f66762-a7fe-46ab-8521-3677250f43cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:11.646927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f66762-a7fe-46ab-8521-3677250f43cb admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:11.648310 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f66762-a7fe-46ab-8521-3677250f43cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:11.650594 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f66762-a7fe-46ab-8521-3677250f43cb admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:11.676117 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 903/1789] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:11 2022] POST /identity/v3/users => generated 291 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:11.680949 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967aadf8-8755-47de-8f16-7abf64624c0c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:11.693750 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967aadf8-8755-47de-8f16-7abf64624c0c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:11.698799 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277fb90-061c-4e45-b2df-61e98e0dc077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:11.699539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277fb90-061c-4e45-b2df-61e98e0dc077 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:11.699828 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277fb90-061c-4e45-b2df-61e98e0dc077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:11.700108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277fb90-061c-4e45-b2df-61e98e0dc077 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:11.701969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277fb90-061c-4e45-b2df-61e98e0dc077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:11.704771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277fb90-061c-4e45-b2df-61e98e0dc077 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:11.722289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 887/1790] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:11.726306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f66762-a7fe-46ab-8521-3677250f43cb admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:11.738621 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f66762-a7fe-46ab-8521-3677250f43cb admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:11.744320 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852221c5-d4f7-4376-9d3e-caa5446e65b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:11.744991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852221c5-d4f7-4376-9d3e-caa5446e65b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:11.745271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852221c5-d4f7-4376-9d3e-caa5446e65b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:11.745543 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852221c5-d4f7-4376-9d3e-caa5446e65b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:11.746419 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852221c5-d4f7-4376-9d3e-caa5446e65b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:11.749518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852221c5-d4f7-4376-9d3e-caa5446e65b1 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:11.764547 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 904/1791] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:11.769856 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277fb90-061c-4e45-b2df-61e98e0dc077 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:11.782731 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277fb90-061c-4e45-b2df-61e98e0dc077 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:11.789530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85997b-5bc2-40d2-9c08-60d62427111e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:11.790457 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85997b-5bc2-40d2-9c08-60d62427111e admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:11.790748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85997b-5bc2-40d2-9c08-60d62427111e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:11.791083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85997b-5bc2-40d2-9c08-60d62427111e admin admin] PATH_INFO: `/v3/projects/cb50ad6f66484232a8352f4d1838cfee/users/a5f04d32af264d438434a363fef4879e/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:11.820454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85997b-5bc2-40d2-9c08-60d62427111e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb50ad6f66484232a8352f4d1838cfee, user_id=a5f04d32af264d438434a363fef4879e, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:11.823220 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85997b-5bc2-40d2-9c08-60d62427111e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:11.837885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 888/1792] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:11 2022] PUT /identity/v3/projects/cb50ad6f66484232a8352f4d1838cfee/users/a5f04d32af264d438434a363fef4879e/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:11.853996 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ac226-8289-4d03-b53f-7e65aafd98e6 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:11.854521 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ac226-8289-4d03-b53f-7e65aafd98e6 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:11.854967 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ac226-8289-4d03-b53f-7e65aafd98e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:11.900611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-cd7ac226-8289-4d03-b53f-7e65aafd98e6 None None] MFA Rules not processed for user `a5f04d32af264d438434a363fef4879e`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:11.901869 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-cd7ac226-8289-4d03-b53f-7e65aafd98e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:11.937509 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:12.004218 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 905/1793] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:12.008686 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85997b-5bc2-40d2-9c08-60d62427111e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:12.119144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85997b-5bc2-40d2-9c08-60d62427111e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:12.132515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b68d26-0de9-48fb-ba6d-1f1bf99e86e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:12.133004 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b68d26-0de9-48fb-ba6d-1f1bf99e86e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:12.133287 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b68d26-0de9-48fb-ba6d-1f1bf99e86e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:12.133561 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b68d26-0de9-48fb-ba6d-1f1bf99e86e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:12.134260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b68d26-0de9-48fb-ba6d-1f1bf99e86e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:12.136238 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b68d26-0de9-48fb-ba6d-1f1bf99e86e0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:12.154490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:12.207788 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 889/1794] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:12 2022] POST /identity/v3/projects => generated 383 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:12.211765 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ac226-8289-4d03-b53f-7e65aafd98e6 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:12.224989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ac226-8289-4d03-b53f-7e65aafd98e6 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:12.308785 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eeed11-d8e8-421f-8cdd-1836c445c4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:12.309729 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eeed11-d8e8-421f-8cdd-1836c445c4f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:12.310093 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eeed11-d8e8-421f-8cdd-1836c445c4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:12.310449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eeed11-d8e8-421f-8cdd-1836c445c4f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:12.311274 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eeed11-d8e8-421f-8cdd-1836c445c4f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:12.313310 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eeed11-d8e8-421f-8cdd-1836c445c4f1 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:12.344804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 906/1795] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:12 2022] POST /identity/v3/users => generated 298 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:12.348490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b68d26-0de9-48fb-ba6d-1f1bf99e86e0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:12.367204 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b68d26-0de9-48fb-ba6d-1f1bf99e86e0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:12.373819 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60a1f2-9765-460f-b9c1-4080c123420f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:12.375011 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60a1f2-9765-460f-b9c1-4080c123420f admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:12.375528 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60a1f2-9765-460f-b9c1-4080c123420f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:12.376136 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60a1f2-9765-460f-b9c1-4080c123420f admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:12.377250 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60a1f2-9765-460f-b9c1-4080c123420f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:12.380035 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60a1f2-9765-460f-b9c1-4080c123420f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:12.398749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 890/1796] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:12.402893 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eeed11-d8e8-421f-8cdd-1836c445c4f1 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:12.417694 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eeed11-d8e8-421f-8cdd-1836c445c4f1 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:12.426707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5311eb6-9dfa-4345-80f2-361a50498279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:12.428181 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5311eb6-9dfa-4345-80f2-361a50498279 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:12.428602 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5311eb6-9dfa-4345-80f2-361a50498279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:12.429863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5311eb6-9dfa-4345-80f2-361a50498279 admin admin] PATH_INFO: `/v3/projects/939cf65f9b8e4533aaf92fbfa0802767/users/27d81ac0cac64bbba4763cc3429edf4e/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:12.461292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5311eb6-9dfa-4345-80f2-361a50498279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=939cf65f9b8e4533aaf92fbfa0802767, user_id=27d81ac0cac64bbba4763cc3429edf4e, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:12.464433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5311eb6-9dfa-4345-80f2-361a50498279 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:12.491536 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 907/1797] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:12 2022] PUT /identity/v3/projects/939cf65f9b8e4533aaf92fbfa0802767/users/27d81ac0cac64bbba4763cc3429edf4e/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:12.496066 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60a1f2-9765-460f-b9c1-4080c123420f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:12.513083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60a1f2-9765-460f-b9c1-4080c123420f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:12.594226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003542b-344d-43f0-a561-54bc675b2965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:12.595112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003542b-344d-43f0-a561-54bc675b2965 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:12.595295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003542b-344d-43f0-a561-54bc675b2965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:12.595539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003542b-344d-43f0-a561-54bc675b2965 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:12.597039 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003542b-344d-43f0-a561-54bc675b2965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:12.598695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003542b-344d-43f0-a561-54bc675b2965 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:12.611729 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 891/1798] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:12.616639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5311eb6-9dfa-4345-80f2-361a50498279 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:12.630777 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5311eb6-9dfa-4345-80f2-361a50498279 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:12.644684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec3bd5-9c5e-45aa-9a1e-a57df01913e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:12.645699 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec3bd5-9c5e-45aa-9a1e-a57df01913e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:12.646060 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec3bd5-9c5e-45aa-9a1e-a57df01913e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:12.646342 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec3bd5-9c5e-45aa-9a1e-a57df01913e8 admin admin] PATH_INFO: `/v3/projects/939cf65f9b8e4533aaf92fbfa0802767/users/27d81ac0cac64bbba4763cc3429edf4e/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:12.649906 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec3bd5-9c5e-45aa-9a1e-a57df01913e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=939cf65f9b8e4533aaf92fbfa0802767, user_id=27d81ac0cac64bbba4763cc3429edf4e, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:12.666518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec3bd5-9c5e-45aa-9a1e-a57df01913e8 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:12.676126 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 908/1799] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:12 2022] PUT /identity/v3/projects/939cf65f9b8e4533aaf92fbfa0802767/users/27d81ac0cac64bbba4763cc3429edf4e/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:12.688510 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2aa1ea-5c00-415f-9790-83c39139aa6d None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:12.688882 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2aa1ea-5c00-415f-9790-83c39139aa6d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:12.689168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2aa1ea-5c00-415f-9790-83c39139aa6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:12.722077 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-5f2aa1ea-5c00-415f-9790-83c39139aa6d None None] MFA Rules not processed for user `27d81ac0cac64bbba4763cc3429edf4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:12.723302 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5f2aa1ea-5c00-415f-9790-83c39139aa6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:12.875951 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 892/1800] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:12.891886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec3bd5-9c5e-45aa-9a1e-a57df01913e8 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:12.894745 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d4ec3bd5-9c5e-45aa-9a1e-a57df01913e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:13.061131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec3bd5-9c5e-45aa-9a1e-a57df01913e8 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:13.073283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b9d68-60ae-4285-b3ab-956d050b1fa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:13.074385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b9d68-60ae-4285-b3ab-956d050b1fa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:13.074826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b9d68-60ae-4285-b3ab-956d050b1fa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:13.075451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b9d68-60ae-4285-b3ab-956d050b1fa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:13.093106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b9d68-60ae-4285-b3ab-956d050b1fa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:13.094678 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b9d68-60ae-4285-b3ab-956d050b1fa8 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:13.107650 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 909/1801] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:35:13.246466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2aa1ea-5c00-415f-9790-83c39139aa6d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:13.247603 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5f2aa1ea-5c00-415f-9790-83c39139aa6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:13.264409 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2aa1ea-5c00-415f-9790-83c39139aa6d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:13.268884 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daafd03-f385-4e2a-afb6-21e499098867 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:13.269682 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daafd03-f385-4e2a-afb6-21e499098867 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:13.270067 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daafd03-f385-4e2a-afb6-21e499098867 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:13.270446 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daafd03-f385-4e2a-afb6-21e499098867 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:13.338447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:13.343137 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daafd03-f385-4e2a-afb6-21e499098867 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:13.344825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daafd03-f385-4e2a-afb6-21e499098867 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:13.364652 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 893/1802] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:17.649934 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b9d68-60ae-4285-b3ab-956d050b1fa8 service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:17.720398 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:17.817779 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b9d68-60ae-4285-b3ab-956d050b1fa8 service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:17.826807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b08e83-8374-4ec7-841f-987e21c279ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:17.829279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b08e83-8374-4ec7-841f-987e21c279ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:17.829576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b08e83-8374-4ec7-841f-987e21c279ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:17.829861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b08e83-8374-4ec7-841f-987e21c279ac admin admin] PATH_INFO: `/v3/users/a5f04d32af264d438434a363fef4879e` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:17.836070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b08e83-8374-4ec7-841f-987e21c279ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5f04d32af264d438434a363fef4879e)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:17.847361 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b08e83-8374-4ec7-841f-987e21c279ac admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:17.998202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-47b08e83-8374-4ec7-841f-987e21c279ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5f04d32af264d438434a363fef4879e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:18.011932 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-47b08e83-8374-4ec7-841f-987e21c279ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5f04d32af264d438434a363fef4879e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:18.031555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-47b08e83-8374-4ec7-841f-987e21c279ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5f04d32af264d438434a363fef4879e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:18.032680 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-47b08e83-8374-4ec7-841f-987e21c279ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5f04d32af264d438434a363fef4879e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:18.034303 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-47b08e83-8374-4ec7-841f-987e21c279ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5f04d32af264d438434a363fef4879e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:18.083897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 910/1803] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:17 2022] DELETE /identity/v3/users/a5f04d32af264d438434a363fef4879e => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:18.086532 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0119e-98d4-42ba-8c88-c0d542569cb3 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:18.088291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0119e-98d4-42ba-8c88-c0d542569cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:18.088852 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0119e-98d4-42ba-8c88-c0d542569cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:18.115910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e4e0119e-98d4-42ba-8c88-c0d542569cb3 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:18.119719 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e4e0119e-98d4-42ba-8c88-c0d542569cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:18.220801 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 894/1804] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:35:18.240631 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b08e83-8374-4ec7-841f-987e21c279ac admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:18.242264 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-47b08e83-8374-4ec7-841f-987e21c279ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:18.399060 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b08e83-8374-4ec7-841f-987e21c279ac admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:18.403318 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f897617-8dd2-4825-bb6f-08e4bf5e69b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:18.406247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f897617-8dd2-4825-bb6f-08e4bf5e69b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:18.406587 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f897617-8dd2-4825-bb6f-08e4bf5e69b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:18.406910 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f897617-8dd2-4825-bb6f-08e4bf5e69b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:18.425278 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f897617-8dd2-4825-bb6f-08e4bf5e69b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:18.426749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f897617-8dd2-4825-bb6f-08e4bf5e69b4 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:18.438960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 911/1805] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:18.861422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0119e-98d4-42ba-8c88-c0d542569cb3 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:18.865751 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e4e0119e-98d4-42ba-8c88-c0d542569cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:18.882930 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0119e-98d4-42ba-8c88-c0d542569cb3 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:18.891258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce4434-f225-4a27-9757-8a298b20de4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:18.893748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce4434-f225-4a27-9757-8a298b20de4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:18.894429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce4434-f225-4a27-9757-8a298b20de4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:18.895072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce4434-f225-4a27-9757-8a298b20de4e admin admin] PATH_INFO: `/v3/projects/cb50ad6f66484232a8352f4d1838cfee` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:18.897995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce4434-f225-4a27-9757-8a298b20de4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb50ad6f66484232a8352f4d1838cfee)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:18.900984 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce4434-f225-4a27-9757-8a298b20de4e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:18.931069 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f897617-8dd2-4825-bb6f-08e4bf5e69b4 service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:18.933087 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-8f897617-8dd2-4825-bb6f-08e4bf5e69b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:18.953092 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f897617-8dd2-4825-bb6f-08e4bf5e69b4 service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:18.976271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e3ce4434-f225-4a27-9757-8a298b20de4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb50ad6f66484232a8352f4d1838cfee', 'request_id': 'req-e3ce4434-f225-4a27-9757-8a298b20de4e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:18.983897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e3ce4434-f225-4a27-9757-8a298b20de4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb50ad6f66484232a8352f4d1838cfee', 'request_id': 'req-e3ce4434-f225-4a27-9757-8a298b20de4e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:18.995571 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e3ce4434-f225-4a27-9757-8a298b20de4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb50ad6f66484232a8352f4d1838cfee', 'request_id': 'req-e3ce4434-f225-4a27-9757-8a298b20de4e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:19.011768 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e3ce4434-f225-4a27-9757-8a298b20de4e admin admin] The token cache is being invalidate because project cb50ad6f66484232a8352f4d1838cfee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:19.012113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e3ce4434-f225-4a27-9757-8a298b20de4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:19.014979 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e3ce4434-f225-4a27-9757-8a298b20de4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:19.020191 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 895/1806] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:18 2022] DELETE /identity/v3/projects/cb50ad6f66484232a8352f4d1838cfee => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:19.021055 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce4434-f225-4a27-9757-8a298b20de4e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:19.022669 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e3ce4434-f225-4a27-9757-8a298b20de4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:19.044355 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:19.057658 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077c950-414b-4eb9-b604-451e6cad1c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:19.058581 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077c950-414b-4eb9-b604-451e6cad1c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:19.059148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077c950-414b-4eb9-b604-451e6cad1c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:19.059585 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077c950-414b-4eb9-b604-451e6cad1c82 admin admin] PATH_INFO: `/v3/users/7a939cb235814ff49d29230c0805d6aa` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:19.061146 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7077c950-414b-4eb9-b604-451e6cad1c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a939cb235814ff49d29230c0805d6aa)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:19.063159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7077c950-414b-4eb9-b604-451e6cad1c82 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:19.109747 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce4434-f225-4a27-9757-8a298b20de4e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:19.115741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd93c8-0961-4e9b-b3e3-029532523369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:19.116638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd93c8-0961-4e9b-b3e3-029532523369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:19.116919 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd93c8-0961-4e9b-b3e3-029532523369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:19.117534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd93c8-0961-4e9b-b3e3-029532523369 admin admin] PATH_INFO: `/v3/users/27d81ac0cac64bbba4763cc3429edf4e` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:19.118951 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd93c8-0961-4e9b-b3e3-029532523369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27d81ac0cac64bbba4763cc3429edf4e)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:19.120898 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd93c8-0961-4e9b-b3e3-029532523369 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:19.124494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7077c950-414b-4eb9-b604-451e6cad1c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a939cb235814ff49d29230c0805d6aa'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:19.133798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7077c950-414b-4eb9-b604-451e6cad1c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a939cb235814ff49d29230c0805d6aa'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:19.143352 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:19.143637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7077c950-414b-4eb9-b604-451e6cad1c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a939cb235814ff49d29230c0805d6aa'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:19.144543 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7077c950-414b-4eb9-b604-451e6cad1c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a939cb235814ff49d29230c0805d6aa'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:19.145146 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7077c950-414b-4eb9-b604-451e6cad1c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a939cb235814ff49d29230c0805d6aa'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:19.169804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 912/1807] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:18 2022] DELETE /identity/v3/users/7a939cb235814ff49d29230c0805d6aa => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:19.171100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077c950-414b-4eb9-b604-451e6cad1c82 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:19.172753 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-7077c950-414b-4eb9-b604-451e6cad1c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:19.180652 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2cdd93c8-0961-4e9b-b3e3-029532523369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27d81ac0cac64bbba4763cc3429edf4e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:19.190788 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2cdd93c8-0961-4e9b-b3e3-029532523369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27d81ac0cac64bbba4763cc3429edf4e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:19.191556 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2cdd93c8-0961-4e9b-b3e3-029532523369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27d81ac0cac64bbba4763cc3429edf4e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:19.192191 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2cdd93c8-0961-4e9b-b3e3-029532523369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27d81ac0cac64bbba4763cc3429edf4e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:19.207317 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2cdd93c8-0961-4e9b-b3e3-029532523369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27d81ac0cac64bbba4763cc3429edf4e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:19.227666 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 896/1808] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:19 2022] DELETE /identity/v3/users/27d81ac0cac64bbba4763cc3429edf4e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:19.227666 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82152b-ea39-4c33-ae67-96a07996261b None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:19.227666 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82152b-ea39-4c33-ae67-96a07996261b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:19.227666 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82152b-ea39-4c33-ae67-96a07996261b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:19.246986 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1b82152b-ea39-4c33-ae67-96a07996261b None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:19.249031 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1b82152b-ea39-4c33-ae67-96a07996261b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:19.258486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077c950-414b-4eb9-b604-451e6cad1c82 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:19.321962 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 897/1809] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:19.332505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807da7f-e908-490c-aa66-ba869dab0cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:19.332697 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82152b-ea39-4c33-ae67-96a07996261b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:19.333400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807da7f-e908-490c-aa66-ba869dab0cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:19.334043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807da7f-e908-490c-aa66-ba869dab0cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:19.334043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807da7f-e908-490c-aa66-ba869dab0cb9 admin admin] PATH_INFO: `/v3/users/cc19db89b395477ba352b9c587822405` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:19.335026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1b82152b-ea39-4c33-ae67-96a07996261b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:19.335477 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807da7f-e908-490c-aa66-ba869dab0cb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc19db89b395477ba352b9c587822405)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:19.337398 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807da7f-e908-490c-aa66-ba869dab0cb9 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:19.363443 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:19.406730 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6807da7f-e908-490c-aa66-ba869dab0cb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc19db89b395477ba352b9c587822405'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:19.408382 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82152b-ea39-4c33-ae67-96a07996261b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:19.412973 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31957179-dd86-4223-bb48-4318ad2c2674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:19.413666 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31957179-dd86-4223-bb48-4318ad2c2674 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:19.414503 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31957179-dd86-4223-bb48-4318ad2c2674 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:19.414833 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31957179-dd86-4223-bb48-4318ad2c2674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:19.417482 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6807da7f-e908-490c-aa66-ba869dab0cb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc19db89b395477ba352b9c587822405'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:19.427380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31957179-dd86-4223-bb48-4318ad2c2674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:19.428405 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6807da7f-e908-490c-aa66-ba869dab0cb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc19db89b395477ba352b9c587822405'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:19.429218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31957179-dd86-4223-bb48-4318ad2c2674 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:19.429337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6807da7f-e908-490c-aa66-ba869dab0cb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc19db89b395477ba352b9c587822405'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:19.430094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6807da7f-e908-490c-aa66-ba869dab0cb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc19db89b395477ba352b9c587822405'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:19.442553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 898/1810] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:35:19.452835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 913/1811] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:19 2022] DELETE /identity/v3/users/cc19db89b395477ba352b9c587822405 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:19.458097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff146a31-fee0-43a8-8d7f-9d585714dfb9 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:19.458376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff146a31-fee0-43a8-8d7f-9d585714dfb9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:19.458685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff146a31-fee0-43a8-8d7f-9d585714dfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:19.476470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ff146a31-fee0-43a8-8d7f-9d585714dfb9 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:19.477810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ff146a31-fee0-43a8-8d7f-9d585714dfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:19.517039 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:19.562540 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 899/1812] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:19.572241 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807da7f-e908-490c-aa66-ba869dab0cb9 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:19.573713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6807da7f-e908-490c-aa66-ba869dab0cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:19.647356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807da7f-e908-490c-aa66-ba869dab0cb9 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:19.651112 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a5401-82e1-4997-96f7-f5460c7b5e2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:19.651857 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a5401-82e1-4997-96f7-f5460c7b5e2e service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:19.652205 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a5401-82e1-4997-96f7-f5460c7b5e2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:19.652647 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a5401-82e1-4997-96f7-f5460c7b5e2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:19.662670 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a5401-82e1-4997-96f7-f5460c7b5e2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:19.664181 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a5401-82e1-4997-96f7-f5460c7b5e2e service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:19.675021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 914/1813] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:19.968066 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff146a31-fee0-43a8-8d7f-9d585714dfb9 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:19.970288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ff146a31-fee0-43a8-8d7f-9d585714dfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:19.985177 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff146a31-fee0-43a8-8d7f-9d585714dfb9 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:19.992229 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b9f6a-c653-4678-b9de-a9f20e1c1696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:19.993339 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b9f6a-c653-4678-b9de-a9f20e1c1696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:19.993648 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b9f6a-c653-4678-b9de-a9f20e1c1696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:19.994133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b9f6a-c653-4678-b9de-a9f20e1c1696 admin admin] PATH_INFO: `/v3/projects/c2c8120704e84c6bb83b12980323ff49` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:19.995733 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b9f6a-c653-4678-b9de-a9f20e1c1696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2c8120704e84c6bb83b12980323ff49)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:19.997854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b9f6a-c653-4678-b9de-a9f20e1c1696 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:20.069263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a30b9f6a-c653-4678-b9de-a9f20e1c1696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2c8120704e84c6bb83b12980323ff49', 'request_id': 'req-a30b9f6a-c653-4678-b9de-a9f20e1c1696'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.077175 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a30b9f6a-c653-4678-b9de-a9f20e1c1696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2c8120704e84c6bb83b12980323ff49', 'request_id': 'req-a30b9f6a-c653-4678-b9de-a9f20e1c1696'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.084405 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a30b9f6a-c653-4678-b9de-a9f20e1c1696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2c8120704e84c6bb83b12980323ff49', 'request_id': 'req-a30b9f6a-c653-4678-b9de-a9f20e1c1696'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.099760 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a30b9f6a-c653-4678-b9de-a9f20e1c1696 admin admin] The token cache is being invalidate because project c2c8120704e84c6bb83b12980323ff49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:20.100202 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a30b9f6a-c653-4678-b9de-a9f20e1c1696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.101001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a30b9f6a-c653-4678-b9de-a9f20e1c1696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.106356 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 900/1814] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:19 2022] DELETE /identity/v3/projects/c2c8120704e84c6bb83b12980323ff49 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:20.107311 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a5401-82e1-4997-96f7-f5460c7b5e2e service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:20.119976 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a5401-82e1-4997-96f7-f5460c7b5e2e service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:20.137444 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d3948-91f0-4b3a-8145-6cf7ef3f967b None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:20.138531 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d3948-91f0-4b3a-8145-6cf7ef3f967b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:20.139057 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d3948-91f0-4b3a-8145-6cf7ef3f967b None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:20.140738 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 901/1815] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 14 20:35:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:35:20.152793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a720b9-15fe-4b43-84c0-e23d01f2e20f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:20.153156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a720b9-15fe-4b43-84c0-e23d01f2e20f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:20.153397 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a720b9-15fe-4b43-84c0-e23d01f2e20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:20.175088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d7a720b9-15fe-4b43-84c0-e23d01f2e20f None None] MFA Rules not processed for user `aa3bb9f1e24a4e9ebd460e97b3736bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:20.176705 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d7a720b9-15fe-4b43-84c0-e23d01f2e20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:20.202914 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca43e8d0-a1da-448c-81c9-f3becc374b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:20.204306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43e8d0-a1da-448c-81c9-f3becc374b87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:20.204801 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43e8d0-a1da-448c-81c9-f3becc374b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:20.205328 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43e8d0-a1da-448c-81c9-f3becc374b87 admin admin] PATH_INFO: `/v3/users/100be68f366948b5ac4ea626ba7ef30a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:20.206983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca43e8d0-a1da-448c-81c9-f3becc374b87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=100be68f366948b5ac4ea626ba7ef30a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:20.209275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca43e8d0-a1da-448c-81c9-f3becc374b87 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:20.255733 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 902/1816] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 14 20:35:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:20.276373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:20.279532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ca43e8d0-a1da-448c-81c9-f3becc374b87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '100be68f366948b5ac4ea626ba7ef30a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.293370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ca43e8d0-a1da-448c-81c9-f3becc374b87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '100be68f366948b5ac4ea626ba7ef30a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.297585 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a720b9-15fe-4b43-84c0-e23d01f2e20f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:20.300131 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d7a720b9-15fe-4b43-84c0-e23d01f2e20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:20.314060 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ca43e8d0-a1da-448c-81c9-f3becc374b87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '100be68f366948b5ac4ea626ba7ef30a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.315041 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ca43e8d0-a1da-448c-81c9-f3becc374b87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '100be68f366948b5ac4ea626ba7ef30a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.315889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ca43e8d0-a1da-448c-81c9-f3becc374b87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '100be68f366948b5ac4ea626ba7ef30a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.323160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a720b9-15fe-4b43-84c0-e23d01f2e20f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:20.331000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fcd319-bab7-41c1-a113-466e65a35bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:20.332014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcd319-bab7-41c1-a113-466e65a35bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:20.332911 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcd319-bab7-41c1-a113-466e65a35bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:20.333941 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcd319-bab7-41c1-a113-466e65a35bf9 admin admin] PATH_INFO: `/v3/projects/939cf65f9b8e4533aaf92fbfa0802767` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:20.335684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fcd319-bab7-41c1-a113-466e65a35bf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=939cf65f9b8e4533aaf92fbfa0802767)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:20.338822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fcd319-bab7-41c1-a113-466e65a35bf9 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:20.363304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 915/1817] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:20 2022] DELETE /identity/v3/users/100be68f366948b5ac4ea626ba7ef30a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:35:20.441121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-39fcd319-bab7-41c1-a113-466e65a35bf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '939cf65f9b8e4533aaf92fbfa0802767', 'request_id': 'req-39fcd319-bab7-41c1-a113-466e65a35bf9'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.442437 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:20.454139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-39fcd319-bab7-41c1-a113-466e65a35bf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '939cf65f9b8e4533aaf92fbfa0802767', 'request_id': 'req-39fcd319-bab7-41c1-a113-466e65a35bf9'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.466786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-39fcd319-bab7-41c1-a113-466e65a35bf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '939cf65f9b8e4533aaf92fbfa0802767', 'request_id': 'req-39fcd319-bab7-41c1-a113-466e65a35bf9'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.500450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-39fcd319-bab7-41c1-a113-466e65a35bf9 admin admin] The token cache is being invalidate because project 939cf65f9b8e4533aaf92fbfa0802767 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:20.500824 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-39fcd319-bab7-41c1-a113-466e65a35bf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.501653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-39fcd319-bab7-41c1-a113-466e65a35bf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.507075 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 903/1818] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:20 2022] DELETE /identity/v3/projects/939cf65f9b8e4533aaf92fbfa0802767 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:35:20.512180 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff14468-ffaf-44bc-82a2-43622c7e0c8d None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:20.512180 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff14468-ffaf-44bc-82a2-43622c7e0c8d None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:20.512403 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff14468-ffaf-44bc-82a2-43622c7e0c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:20.555742 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-aff14468-ffaf-44bc-82a2-43622c7e0c8d None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:20.557243 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-aff14468-ffaf-44bc-82a2-43622c7e0c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:20.715017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 916/1819] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:20.722557 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e7d74-6d9f-47a0-8e33-2404dd62f5ba None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:20.722854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e7d74-6d9f-47a0-8e33-2404dd62f5ba None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:20.725207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e7d74-6d9f-47a0-8e33-2404dd62f5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:20.746570 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f79e7d74-6d9f-47a0-8e33-2404dd62f5ba None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:20.748364 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f79e7d74-6d9f-47a0-8e33-2404dd62f5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:20.758278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 904/1820] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:20.762610 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff14468-ffaf-44bc-82a2-43622c7e0c8d None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:20.781463 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff14468-ffaf-44bc-82a2-43622c7e0c8d None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:20.791191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee5dd1-4ad5-4528-909f-8895afd8c3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:20.792397 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee5dd1-4ad5-4528-909f-8895afd8c3e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:20.792843 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee5dd1-4ad5-4528-909f-8895afd8c3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:20.793300 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee5dd1-4ad5-4528-909f-8895afd8c3e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:20.794439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee5dd1-4ad5-4528-909f-8895afd8c3e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:35:20.794764 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee5dd1-4ad5-4528-909f-8895afd8c3e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:20.796949 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee5dd1-4ad5-4528-909f-8895afd8c3e4 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:20.804231 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e7d74-6d9f-47a0-8e33-2404dd62f5ba None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:20.806113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f79e7d74-6d9f-47a0-8e33-2404dd62f5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:20.829983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e7d74-6d9f-47a0-8e33-2404dd62f5ba None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:20.836285 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b43b53-7401-4479-abd9-6688873e2348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:20.837452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b43b53-7401-4479-abd9-6688873e2348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:20.837818 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b43b53-7401-4479-abd9-6688873e2348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:20.838201 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b43b53-7401-4479-abd9-6688873e2348 admin admin] PATH_INFO: `/v3/projects/4e3c60bbad5a4e72a90e3e014105466e` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:20.838386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 917/1821] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:35:20.839467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee5dd1-4ad5-4528-909f-8895afd8c3e4 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:20.842334 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b43b53-7401-4479-abd9-6688873e2348 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e3c60bbad5a4e72a90e3e014105466e)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:20.843754 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f3ee5dd1-4ad5-4528-909f-8895afd8c3e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:20.845558 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b43b53-7401-4479-abd9-6688873e2348 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:20.909309 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-48b43b53-7401-4479-abd9-6688873e2348 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e3c60bbad5a4e72a90e3e014105466e', 'request_id': 'req-48b43b53-7401-4479-abd9-6688873e2348'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.928807 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-48b43b53-7401-4479-abd9-6688873e2348 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e3c60bbad5a4e72a90e3e014105466e', 'request_id': 'req-48b43b53-7401-4479-abd9-6688873e2348'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.942494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-48b43b53-7401-4479-abd9-6688873e2348 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e3c60bbad5a4e72a90e3e014105466e', 'request_id': 'req-48b43b53-7401-4479-abd9-6688873e2348'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.961828 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-48b43b53-7401-4479-abd9-6688873e2348 admin admin] The token cache is being invalidate because project 4e3c60bbad5a4e72a90e3e014105466e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:20.962445 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-48b43b53-7401-4479-abd9-6688873e2348 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.963489 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-48b43b53-7401-4479-abd9-6688873e2348 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:20.971101 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 905/1822] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:20 2022] DELETE /identity/v3/projects/4e3c60bbad5a4e72a90e3e014105466e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:20.971101 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b43b53-7401-4479-abd9-6688873e2348 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:20.971101 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-48b43b53-7401-4479-abd9-6688873e2348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:20.975461 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee5dd1-4ad5-4528-909f-8895afd8c3e4 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:21.052208 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b43b53-7401-4479-abd9-6688873e2348 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:21.056253 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:21.057293 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335503a4-bcec-419b-a39c-23f89c04f5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:21.058063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335503a4-bcec-419b-a39c-23f89c04f5c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.058434 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335503a4-bcec-419b-a39c-23f89c04f5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.058610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335503a4-bcec-419b-a39c-23f89c04f5c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:21.059531 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335503a4-bcec-419b-a39c-23f89c04f5c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:21.061525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335503a4-bcec-419b-a39c-23f89c04f5c7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:21.081432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05a3e4-703b-4657-897b-eab09f7b10b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:21.082443 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05a3e4-703b-4657-897b-eab09f7b10b0 service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.082728 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05a3e4-703b-4657-897b-eab09f7b10b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.083037 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05a3e4-703b-4657-897b-eab09f7b10b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:21.085217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-7b05a3e4-703b-4657-897b-eab09f7b10b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:21.093312 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 906/1823] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:20 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:21.093312 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335503a4-bcec-419b-a39c-23f89c04f5c7 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:21.095415 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-335503a4-bcec-419b-a39c-23f89c04f5c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:21.098567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b05a3e4-703b-4657-897b-eab09f7b10b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:21.100229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b05a3e4-703b-4657-897b-eab09f7b10b0 service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:21.110979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 918/1824] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:21.112292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef10a0-1c20-4e71-8359-e9eb5a1f9f03 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.112585 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef10a0-1c20-4e71-8359-e9eb5a1f9f03 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.112871 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef10a0-1c20-4e71-8359-e9eb5a1f9f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:21.132020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-93ef10a0-1c20-4e71-8359-e9eb5a1f9f03 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:21.133324 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-93ef10a0-1c20-4e71-8359-e9eb5a1f9f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:21.138648 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:21.163423 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335503a4-bcec-419b-a39c-23f89c04f5c7 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:21.168201 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24f420-b6bb-487c-9065-a68c6cbbd674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:21.169060 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24f420-b6bb-487c-9065-a68c6cbbd674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.169361 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24f420-b6bb-487c-9065-a68c6cbbd674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.169604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24f420-b6bb-487c-9065-a68c6cbbd674 admin admin] PATH_INFO: `/v3/projects/dc77ea5b3a3f4cfcb180eac4fae6badb` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:21.171150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24f420-b6bb-487c-9065-a68c6cbbd674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc77ea5b3a3f4cfcb180eac4fae6badb)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:21.173318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24f420-b6bb-487c-9065-a68c6cbbd674 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:21.199972 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 919/1825] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:21.201147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef10a0-1c20-4e71-8359-e9eb5a1f9f03 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:21.211866 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef10a0-1c20-4e71-8359-e9eb5a1f9f03 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:21.223277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4b24f420-b6bb-487c-9065-a68c6cbbd674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc77ea5b3a3f4cfcb180eac4fae6badb', 'request_id': 'req-4b24f420-b6bb-487c-9065-a68c6cbbd674'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:21.229603 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4b24f420-b6bb-487c-9065-a68c6cbbd674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc77ea5b3a3f4cfcb180eac4fae6badb', 'request_id': 'req-4b24f420-b6bb-487c-9065-a68c6cbbd674'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:21.235101 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4b24f420-b6bb-487c-9065-a68c6cbbd674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc77ea5b3a3f4cfcb180eac4fae6badb', 'request_id': 'req-4b24f420-b6bb-487c-9065-a68c6cbbd674'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:21.248913 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4b24f420-b6bb-487c-9065-a68c6cbbd674 admin admin] The token cache is being invalidate because project dc77ea5b3a3f4cfcb180eac4fae6badb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:21.249205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4b24f420-b6bb-487c-9065-a68c6cbbd674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:21.249855 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4b24f420-b6bb-487c-9065-a68c6cbbd674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:21.251481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 907/1826] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:21 2022] DELETE /identity/v3/projects/dc77ea5b3a3f4cfcb180eac4fae6badb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:21.253281 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c63d22-9c58-4d52-8f7a-c4248bb36ed4 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.253569 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c63d22-9c58-4d52-8f7a-c4248bb36ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.253849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c63d22-9c58-4d52-8f7a-c4248bb36ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:21.272235 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a2c63d22-9c58-4d52-8f7a-c4248bb36ed4 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:21.272414 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c02b7-3d64-4428-9358-f34f85ade93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:21.272995 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c02b7-3d64-4428-9358-f34f85ade93d admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.273284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c02b7-3d64-4428-9358-f34f85ade93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.273453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a2c63d22-9c58-4d52-8f7a-c4248bb36ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:21.273563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c02b7-3d64-4428-9358-f34f85ade93d admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:21.274254 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c02b7-3d64-4428-9358-f34f85ade93d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:21.276217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c02b7-3d64-4428-9358-f34f85ade93d admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:21.281529 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 908/1827] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:21.282310 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c63d22-9c58-4d52-8f7a-c4248bb36ed4 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:21.283669 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a2c63d22-9c58-4d52-8f7a-c4248bb36ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:21.296887 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c63d22-9c58-4d52-8f7a-c4248bb36ed4 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:21.298485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 920/1828] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:21 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:21.299686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c02b7-3d64-4428-9358-f34f85ade93d admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:21.303627 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c01f6-041b-4d44-9e66-9cfdd1b4d1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:21.304532 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c01f6-041b-4d44-9e66-9cfdd1b4d1fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.304816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c01f6-041b-4d44-9e66-9cfdd1b4d1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.305130 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c01f6-041b-4d44-9e66-9cfdd1b4d1fd admin admin] PATH_INFO: `/v3/users/89f6ae40173e404aa4e5c5e761fddd3f` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:21.306481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c01f6-041b-4d44-9e66-9cfdd1b4d1fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=89f6ae40173e404aa4e5c5e761fddd3f)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:21.308476 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c01f6-041b-4d44-9e66-9cfdd1b4d1fd admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:21.310787 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c02b7-3d64-4428-9358-f34f85ade93d admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:21.316274 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b036cb-da4b-41d3-88c4-404ce2384800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:21.316931 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b036cb-da4b-41d3-88c4-404ce2384800 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.317215 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b036cb-da4b-41d3-88c4-404ce2384800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.317498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b036cb-da4b-41d3-88c4-404ce2384800 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:21.318231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b036cb-da4b-41d3-88c4-404ce2384800 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:35:21.318542 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b036cb-da4b-41d3-88c4-404ce2384800 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:21.320179 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b036cb-da4b-41d3-88c4-404ce2384800 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:21.332882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 921/1829] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:35:21.333898 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b036cb-da4b-41d3-88c4-404ce2384800 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:21.335991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-93b036cb-da4b-41d3-88c4-404ce2384800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:21.349051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b036cb-da4b-41d3-88c4-404ce2384800 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:21.353673 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea17261-95f1-4da2-9518-5d83a14640d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:21.354333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea17261-95f1-4da2-9518-5d83a14640d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.354626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea17261-95f1-4da2-9518-5d83a14640d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.354900 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea17261-95f1-4da2-9518-5d83a14640d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:21.355657 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea17261-95f1-4da2-9518-5d83a14640d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:21.357241 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea17261-95f1-4da2-9518-5d83a14640d1 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:21.358725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-834c01f6-041b-4d44-9e66-9cfdd1b4d1fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89f6ae40173e404aa4e5c5e761fddd3f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:21.366819 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-834c01f6-041b-4d44-9e66-9cfdd1b4d1fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89f6ae40173e404aa4e5c5e761fddd3f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:21.367440 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-834c01f6-041b-4d44-9e66-9cfdd1b4d1fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89f6ae40173e404aa4e5c5e761fddd3f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:21.368049 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-834c01f6-041b-4d44-9e66-9cfdd1b4d1fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89f6ae40173e404aa4e5c5e761fddd3f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:21.368310 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 922/1830] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:21.371521 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea17261-95f1-4da2-9518-5d83a14640d1 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:21.372679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-2ea17261-95f1-4da2-9518-5d83a14640d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:21.382227 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-834c01f6-041b-4d44-9e66-9cfdd1b4d1fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89f6ae40173e404aa4e5c5e761fddd3f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:21.386718 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea17261-95f1-4da2-9518-5d83a14640d1 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:21.392489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2ec2e-b971-4c0c-a3ff-6172ad7ccacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:21.393372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2ec2e-b971-4c0c-a3ff-6172ad7ccacf admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.393661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2ec2e-b971-4c0c-a3ff-6172ad7ccacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.393935 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2ec2e-b971-4c0c-a3ff-6172ad7ccacf admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:21.394844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2ec2e-b971-4c0c-a3ff-6172ad7ccacf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:21.397420 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2ec2e-b971-4c0c-a3ff-6172ad7ccacf admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:21.412738 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 909/1831] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:21 2022] DELETE /identity/v3/users/89f6ae40173e404aa4e5c5e761fddd3f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:21.412738 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c01f6-041b-4d44-9e66-9cfdd1b4d1fd admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:21.412738 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-834c01f6-041b-4d44-9e66-9cfdd1b4d1fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:21.491031 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 923/1832] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:21 2022] POST /identity/v3/projects => generated 393 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:21.491031 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2ec2e-b971-4c0c-a3ff-6172ad7ccacf admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:21.523410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:21.528239 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:21.580266 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c01f6-041b-4d44-9e66-9cfdd1b4d1fd admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:21.586925 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a33413-12be-419f-9063-dd63264b8f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:21.588496 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a33413-12be-419f-9063-dd63264b8f22 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.589164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a33413-12be-419f-9063-dd63264b8f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.589723 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a33413-12be-419f-9063-dd63264b8f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:21.590528 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a33413-12be-419f-9063-dd63264b8f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:21.592419 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a33413-12be-419f-9063-dd63264b8f22 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:21.593206 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2ec2e-b971-4c0c-a3ff-6172ad7ccacf admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:21.600543 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465ac6a-5300-4360-93b2-90e5d1a32d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:21.601391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465ac6a-5300-4360-93b2-90e5d1a32d00 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.601740 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465ac6a-5300-4360-93b2-90e5d1a32d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.602108 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465ac6a-5300-4360-93b2-90e5d1a32d00 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:21.602975 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465ac6a-5300-4360-93b2-90e5d1a32d00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:21.604989 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 910/1833] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:21.605112 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465ac6a-5300-4360-93b2-90e5d1a32d00 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:21.608845 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a33413-12be-419f-9063-dd63264b8f22 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:21.621046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a33413-12be-419f-9063-dd63264b8f22 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:21.627011 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87abe950-ca06-43b9-ab39-14a7fcc12167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:21.627294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abe950-ca06-43b9-ab39-14a7fcc12167 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.630983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abe950-ca06-43b9-ab39-14a7fcc12167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.630983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abe950-ca06-43b9-ab39-14a7fcc12167 admin admin] PATH_INFO: `/v3/projects/780591457f36403183bfb2828ef83e1f/users/a27750452bd34280a9102e093fc042c6/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:21.631188 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 924/1834] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:21 2022] POST /identity/v3/users => generated 297 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:21.636122 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465ac6a-5300-4360-93b2-90e5d1a32d00 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:21.647786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87abe950-ca06-43b9-ab39-14a7fcc12167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=780591457f36403183bfb2828ef83e1f, user_id=a27750452bd34280a9102e093fc042c6, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:21.650890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87abe950-ca06-43b9-ab39-14a7fcc12167 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:21.654616 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465ac6a-5300-4360-93b2-90e5d1a32d00 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:21.660697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b2c63-925f-417f-9aff-f760f0cd4e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:21.661774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b2c63-925f-417f-9aff-f760f0cd4e6d admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.662293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b2c63-925f-417f-9aff-f760f0cd4e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.662783 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b2c63-925f-417f-9aff-f760f0cd4e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:21.663781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b2c63-925f-417f-9aff-f760f0cd4e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:21.666456 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b2c63-925f-417f-9aff-f760f0cd4e6d admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:21.668084 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 911/1835] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:21 2022] PUT /identity/v3/projects/780591457f36403183bfb2828ef83e1f/users/a27750452bd34280a9102e093fc042c6/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:35:21.679802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21498d83-e5b9-4520-a06c-6467fedb30b4 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.680020 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21498d83-e5b9-4520-a06c-6467fedb30b4 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.680439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21498d83-e5b9-4520-a06c-6467fedb30b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:21.689728 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 925/1836] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:35:21.693727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b2c63-925f-417f-9aff-f760f0cd4e6d admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:21.704728 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b2c63-925f-417f-9aff-f760f0cd4e6d admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:21.717327 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-21498d83-e5b9-4520-a06c-6467fedb30b4 None None] MFA Rules not processed for user `a27750452bd34280a9102e093fc042c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:21.718532 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-21498d83-e5b9-4520-a06c-6467fedb30b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:21.808818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8caa2-38e0-4038-8e40-26af3fdfe85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:21.809817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8caa2-38e0-4038-8e40-26af3fdfe85c admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.810107 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8caa2-38e0-4038-8e40-26af3fdfe85c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.810482 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8caa2-38e0-4038-8e40-26af3fdfe85c admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:21.811539 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8caa2-38e0-4038-8e40-26af3fdfe85c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:21.815450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8caa2-38e0-4038-8e40-26af3fdfe85c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:21.828044 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 912/1837] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:21.829215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21498d83-e5b9-4520-a06c-6467fedb30b4 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:21.832862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 926/1838] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:21.838675 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8caa2-38e0-4038-8e40-26af3fdfe85c admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:21.841914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21498d83-e5b9-4520-a06c-6467fedb30b4 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:21.850109 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d54fa-6740-4a8a-b8ab-0200e1528c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:21.850785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d54fa-6740-4a8a-b8ab-0200e1528c35 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.851097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d54fa-6740-4a8a-b8ab-0200e1528c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.851379 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d54fa-6740-4a8a-b8ab-0200e1528c35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:21.852088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d54fa-6740-4a8a-b8ab-0200e1528c35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:21.858723 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8caa2-38e0-4038-8e40-26af3fdfe85c admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:21.864330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fba856-2c1a-4885-9672-3b0ccb7b2250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:21.865245 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fba856-2c1a-4885-9672-3b0ccb7b2250 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.865989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fba856-2c1a-4885-9672-3b0ccb7b2250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.866277 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fba856-2c1a-4885-9672-3b0ccb7b2250 admin admin] PATH_INFO: `/v3/projects/754784e6d35a444481149ea200cdd961/users/2e6d98a7fe104f90a8253cd37a932e79/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:21.870775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d54fa-6740-4a8a-b8ab-0200e1528c35 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:21.925524 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fba856-2c1a-4885-9672-3b0ccb7b2250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=754784e6d35a444481149ea200cdd961, user_id=2e6d98a7fe104f90a8253cd37a932e79, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:21.928665 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fba856-2c1a-4885-9672-3b0ccb7b2250 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:21.953797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 927/1839] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:21 2022] PUT /identity/v3/projects/754784e6d35a444481149ea200cdd961/users/2e6d98a7fe104f90a8253cd37a932e79/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:21.963915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 913/1840] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:21 2022] POST /identity/v3/projects => generated 381 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:21.966801 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5acf89-8c4c-453d-8ad0-5f85fc7f5cb3 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:21.967290 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5acf89-8c4c-453d-8ad0-5f85fc7f5cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:21.967612 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d54fa-6740-4a8a-b8ab-0200e1528c35 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:21.967751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5acf89-8c4c-453d-8ad0-5f85fc7f5cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:22.024672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-6c5acf89-8c4c-453d-8ad0-5f85fc7f5cb3 None None] MFA Rules not processed for user `2e6d98a7fe104f90a8253cd37a932e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:22.029341 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6c5acf89-8c4c-453d-8ad0-5f85fc7f5cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:22.119313 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:22.130359 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d54fa-6740-4a8a-b8ab-0200e1528c35 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:22.135947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 928/1841] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:22.139050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5acf89-8c4c-453d-8ad0-5f85fc7f5cb3 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:22.143631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a88d4-3e0b-41e8-a958-b7fd17b09b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:22.145057 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a88d4-3e0b-41e8-a958-b7fd17b09b4f admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:22.148355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a88d4-3e0b-41e8-a958-b7fd17b09b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:22.148679 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a88d4-3e0b-41e8-a958-b7fd17b09b4f admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:22.149439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a88d4-3e0b-41e8-a958-b7fd17b09b4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:22.152290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a88d4-3e0b-41e8-a958-b7fd17b09b4f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:22.171299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5acf89-8c4c-453d-8ad0-5f85fc7f5cb3 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:22.179438 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cf560-05d8-41ac-a2a7-af645a776dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:22.180769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cf560-05d8-41ac-a2a7-af645a776dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:22.180769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cf560-05d8-41ac-a2a7-af645a776dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:22.181086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cf560-05d8-41ac-a2a7-af645a776dc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:22.182153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40cf560-05d8-41ac-a2a7-af645a776dc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:22.184696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40cf560-05d8-41ac-a2a7-af645a776dc1 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:22.213082 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 914/1842] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:21 2022] POST /identity/v3/users => generated 297 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:35:22.216819 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a88d4-3e0b-41e8-a958-b7fd17b09b4f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:22.231809 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:22.232316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 929/1843] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:22 2022] POST /identity/v3/projects => generated 393 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:22.237934 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cf560-05d8-41ac-a2a7-af645a776dc1 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:22.250202 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a88d4-3e0b-41e8-a958-b7fd17b09b4f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:22.322090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cf560-05d8-41ac-a2a7-af645a776dc1 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:22.333043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70059380-b17d-4693-99ea-9e60ff1dd8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:22.334045 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70059380-b17d-4693-99ea-9e60ff1dd8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:22.334718 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70059380-b17d-4693-99ea-9e60ff1dd8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:22.335858 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70059380-b17d-4693-99ea-9e60ff1dd8b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:22.337562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70059380-b17d-4693-99ea-9e60ff1dd8b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:22.340569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70059380-b17d-4693-99ea-9e60ff1dd8b8 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:22.377373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 930/1844] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:22 2022] POST /identity/v3/users => generated 303 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:22.384841 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70059380-b17d-4693-99ea-9e60ff1dd8b8 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:22.423441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70059380-b17d-4693-99ea-9e60ff1dd8b8 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:22.439281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48e62e-6ea3-4402-a5ea-2c91cbd56169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:22.439951 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a27f74-f0f8-4854-a973-cfda4110e5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:22.440683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a27f74-f0f8-4854-a973-cfda4110e5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:22.441392 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48e62e-6ea3-4402-a5ea-2c91cbd56169 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:22.441687 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48e62e-6ea3-4402-a5ea-2c91cbd56169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:22.441972 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48e62e-6ea3-4402-a5ea-2c91cbd56169 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:22.442733 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48e62e-6ea3-4402-a5ea-2c91cbd56169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:22.448904 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48e62e-6ea3-4402-a5ea-2c91cbd56169 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:22.449278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a27f74-f0f8-4854-a973-cfda4110e5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:22.449578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a27f74-f0f8-4854-a973-cfda4110e5c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:22.450520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a27f74-f0f8-4854-a973-cfda4110e5c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:22.460263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a27f74-f0f8-4854-a973-cfda4110e5c0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:22.492669 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 931/1845] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:22.494403 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48e62e-6ea3-4402-a5ea-2c91cbd56169 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:22.504291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 915/1846] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:35:22.505901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a27f74-f0f8-4854-a973-cfda4110e5c0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:22.513108 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48e62e-6ea3-4402-a5ea-2c91cbd56169 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:22.519765 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea91c56-15ec-43c8-89eb-9d2ab3f0da15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:22.522262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea91c56-15ec-43c8-89eb-9d2ab3f0da15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:22.522378 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea91c56-15ec-43c8-89eb-9d2ab3f0da15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:22.522702 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea91c56-15ec-43c8-89eb-9d2ab3f0da15 admin admin] PATH_INFO: `/v3/projects/40c468289eb244ecae654bebe0b77034/users/e43168af115f42fd8a5938ce23353b39/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:22.535880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a27f74-f0f8-4854-a973-cfda4110e5c0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:22.550464 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5bcff7-a86e-439c-af4b-db238ff7baec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:22.551534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bcff7-a86e-439c-af4b-db238ff7baec admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:22.551892 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bcff7-a86e-439c-af4b-db238ff7baec admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:22.552240 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bcff7-a86e-439c-af4b-db238ff7baec admin admin] PATH_INFO: `/v3/projects/8349d432c68e4fbaab8a3a7fb5b9aec1/users/09b04922eabc4b359eeae2554e81bbd0/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:22.552678 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea91c56-15ec-43c8-89eb-9d2ab3f0da15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40c468289eb244ecae654bebe0b77034, user_id=e43168af115f42fd8a5938ce23353b39, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:22.555817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea91c56-15ec-43c8-89eb-9d2ab3f0da15 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:22.569837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5bcff7-a86e-439c-af4b-db238ff7baec admin admin] RBAC: Authorizing `identity:create_grant(project_id=8349d432c68e4fbaab8a3a7fb5b9aec1, user_id=09b04922eabc4b359eeae2554e81bbd0, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:22.572109 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 932/1847] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:22 2022] PUT /identity/v3/projects/40c468289eb244ecae654bebe0b77034/users/e43168af115f42fd8a5938ce23353b39/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:35:22.572695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5bcff7-a86e-439c-af4b-db238ff7baec admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:22.573381 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea91c56-15ec-43c8-89eb-9d2ab3f0da15 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:22.576418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-1ea91c56-15ec-43c8-89eb-9d2ab3f0da15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:22.590797 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 916/1848] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:22 2022] PUT /identity/v3/projects/8349d432c68e4fbaab8a3a7fb5b9aec1/users/09b04922eabc4b359eeae2554e81bbd0/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:22.592139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5bcff7-a86e-439c-af4b-db238ff7baec admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:22.661794 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea91c56-15ec-43c8-89eb-9d2ab3f0da15 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:22.666645 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c542b8-9f35-4039-b43a-86b2031dafbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:22.667095 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5bcff7-a86e-439c-af4b-db238ff7baec admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:22.667563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c542b8-9f35-4039-b43a-86b2031dafbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:22.667882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c542b8-9f35-4039-b43a-86b2031dafbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:22.668164 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c542b8-9f35-4039-b43a-86b2031dafbd admin admin] PATH_INFO: `/v3/projects/6cab75615bb04e9c86e3b8260f5992b7` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:22.669356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c542b8-9f35-4039-b43a-86b2031dafbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cab75615bb04e9c86e3b8260f5992b7)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:22.671305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c542b8-9f35-4039-b43a-86b2031dafbd admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:22.672008 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f549505-80a3-44f3-9b81-c1f1209e2ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:22.672674 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f549505-80a3-44f3-9b81-c1f1209e2ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:22.672962 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f549505-80a3-44f3-9b81-c1f1209e2ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:22.673242 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f549505-80a3-44f3-9b81-c1f1209e2ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:22.674014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f549505-80a3-44f3-9b81-c1f1209e2ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:22.675650 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f549505-80a3-44f3-9b81-c1f1209e2ee2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:22.687399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 917/1849] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:22.688064 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f549505-80a3-44f3-9b81-c1f1209e2ee2 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:22.699944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f549505-80a3-44f3-9b81-c1f1209e2ee2 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:22.705015 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d0f1b-a27a-4d30-b0c4-416046d2777b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:22.706256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d0f1b-a27a-4d30-b0c4-416046d2777b admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:22.706538 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d0f1b-a27a-4d30-b0c4-416046d2777b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:22.706841 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d0f1b-a27a-4d30-b0c4-416046d2777b admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:22.707621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d0f1b-a27a-4d30-b0c4-416046d2777b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:22.709217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d0f1b-a27a-4d30-b0c4-416046d2777b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:22.709769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:22.724140 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 918/1850] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:22.724140 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d0f1b-a27a-4d30-b0c4-416046d2777b admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:22.728115 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-c8c542b8-9f35-4039-b43a-86b2031dafbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cab75615bb04e9c86e3b8260f5992b7', 'request_id': 'req-c8c542b8-9f35-4039-b43a-86b2031dafbd'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:22.732066 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d0f1b-a27a-4d30-b0c4-416046d2777b admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:22.735864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-c8c542b8-9f35-4039-b43a-86b2031dafbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cab75615bb04e9c86e3b8260f5992b7', 'request_id': 'req-c8c542b8-9f35-4039-b43a-86b2031dafbd'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:22.743226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-c8c542b8-9f35-4039-b43a-86b2031dafbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cab75615bb04e9c86e3b8260f5992b7', 'request_id': 'req-c8c542b8-9f35-4039-b43a-86b2031dafbd'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:22.755791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-c8c542b8-9f35-4039-b43a-86b2031dafbd admin admin] The token cache is being invalidate because project 6cab75615bb04e9c86e3b8260f5992b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:22.756087 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-c8c542b8-9f35-4039-b43a-86b2031dafbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:22.756763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-c8c542b8-9f35-4039-b43a-86b2031dafbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:22.758338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 933/1851] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:22 2022] DELETE /identity/v3/projects/6cab75615bb04e9c86e3b8260f5992b7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:22.759288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c542b8-9f35-4039-b43a-86b2031dafbd admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:22.760602 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-c8c542b8-9f35-4039-b43a-86b2031dafbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:22.809268 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270476d2-abef-40ef-bfef-dfc66180f59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:22.810334 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270476d2-abef-40ef-bfef-dfc66180f59a admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:22.810617 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270476d2-abef-40ef-bfef-dfc66180f59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:22.810890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270476d2-abef-40ef-bfef-dfc66180f59a admin admin] PATH_INFO: `/v3/projects/40c468289eb244ecae654bebe0b77034/users/e43168af115f42fd8a5938ce23353b39/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:22.813613 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270476d2-abef-40ef-bfef-dfc66180f59a admin admin] RBAC: Authorizing `identity:create_grant(project_id=40c468289eb244ecae654bebe0b77034, user_id=e43168af115f42fd8a5938ce23353b39, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:22.816989 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270476d2-abef-40ef-bfef-dfc66180f59a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:22.821357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:22.835781 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 919/1852] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:22 2022] PUT /identity/v3/projects/40c468289eb244ecae654bebe0b77034/users/e43168af115f42fd8a5938ce23353b39/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:22.837802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c737f-8ad6-4e98-892f-860240d48fbe None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:22.838401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c737f-8ad6-4e98-892f-860240d48fbe None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:22.838982 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c737f-8ad6-4e98-892f-860240d48fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:22.858571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c542b8-9f35-4039-b43a-86b2031dafbd admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:22.883861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-415c737f-8ad6-4e98-892f-860240d48fbe None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:22.885247 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-415c737f-8ad6-4e98-892f-860240d48fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:22.938949 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ec41f-6eaf-4342-be90-5d9cec4935fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:22.940665 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ec41f-6eaf-4342-be90-5d9cec4935fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:22.941013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ec41f-6eaf-4342-be90-5d9cec4935fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:22.941301 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ec41f-6eaf-4342-be90-5d9cec4935fe admin admin] PATH_INFO: `/v3/projects/8349d432c68e4fbaab8a3a7fb5b9aec1/users/09b04922eabc4b359eeae2554e81bbd0/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:22.945802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ec41f-6eaf-4342-be90-5d9cec4935fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=8349d432c68e4fbaab8a3a7fb5b9aec1, user_id=09b04922eabc4b359eeae2554e81bbd0, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:22.949928 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ec41f-6eaf-4342-be90-5d9cec4935fe admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:22.965323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 934/1853] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:22 2022] PUT /identity/v3/projects/8349d432c68e4fbaab8a3a7fb5b9aec1/users/09b04922eabc4b359eeae2554e81bbd0/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:22.967489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025bc58-2595-4fcc-ab7d-47609d2c43ba None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:22.968130 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025bc58-2595-4fcc-ab7d-47609d2c43ba None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:22.968638 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025bc58-2595-4fcc-ab7d-47609d2c43ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:22.979103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 920/1854] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:22.980584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5cab4c-a80c-41d5-9022-ee2d56469e68 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:22.981121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5cab4c-a80c-41d5-9022-ee2d56469e68 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:22.981402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5cab4c-a80c-41d5-9022-ee2d56469e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:23.003104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-8025bc58-2595-4fcc-ab7d-47609d2c43ba None None] MFA Rules not processed for user `e43168af115f42fd8a5938ce23353b39`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:23.003104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-8025bc58-2595-4fcc-ab7d-47609d2c43ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:23.014237 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1a5cab4c-a80c-41d5-9022-ee2d56469e68 None None] MFA Rules not processed for user `09b04922eabc4b359eeae2554e81bbd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:23.016637 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1a5cab4c-a80c-41d5-9022-ee2d56469e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:23.072907 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:23.114581 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 935/1855] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:22 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:23.116570 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51432abb-8d65-4191-b410-b688d8ede2a3 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:23.117044 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51432abb-8d65-4191-b410-b688d8ede2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:23.117468 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51432abb-8d65-4191-b410-b688d8ede2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:23.141520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-51432abb-8d65-4191-b410-b688d8ede2a3 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:23.143853 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 921/1856] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:23.144722 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5cab4c-a80c-41d5-9022-ee2d56469e68 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:23.145243 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-51432abb-8d65-4191-b410-b688d8ede2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:23.146128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1a5cab4c-a80c-41d5-9022-ee2d56469e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:23.224634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5cab4c-a80c-41d5-9022-ee2d56469e68 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:23.228887 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655822a6-79d0-4328-8eff-d68ed1e2b8cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:23.229178 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655822a6-79d0-4328-8eff-d68ed1e2b8cb service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:23.229490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655822a6-79d0-4328-8eff-d68ed1e2b8cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:23.229835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655822a6-79d0-4328-8eff-d68ed1e2b8cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:23.242530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655822a6-79d0-4328-8eff-d68ed1e2b8cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:23.244683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655822a6-79d0-4328-8eff-d68ed1e2b8cb service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:23.251739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 936/1857] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:23.253593 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51432abb-8d65-4191-b410-b688d8ede2a3 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:23.256085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-51432abb-8d65-4191-b410-b688d8ede2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:23.271098 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 922/1858] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:23.272777 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51432abb-8d65-4191-b410-b688d8ede2a3 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:23.273956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655822a6-79d0-4328-8eff-d68ed1e2b8cb service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:23.277349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7712c4-162e-4244-aa24-707e5d6f756e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:23.278210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7712c4-162e-4244-aa24-707e5d6f756e service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:23.278663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7712c4-162e-4244-aa24-707e5d6f756e service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:23.279068 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7712c4-162e-4244-aa24-707e5d6f756e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:23.289793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7712c4-162e-4244-aa24-707e5d6f756e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:23.293017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7712c4-162e-4244-aa24-707e5d6f756e service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:23.303747 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655822a6-79d0-4328-8eff-d68ed1e2b8cb service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:23.306311 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 937/1859] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:23.321307 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7712c4-162e-4244-aa24-707e5d6f756e service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:23.323663 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b9de0-6356-40e2-b958-02891a86ea67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:23.325923 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b9de0-6356-40e2-b958-02891a86ea67 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:23.326211 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b9de0-6356-40e2-b958-02891a86ea67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:23.326489 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b9de0-6356-40e2-b958-02891a86ea67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:23.331272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b9de0-6356-40e2-b958-02891a86ea67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:35:23.331969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b9de0-6356-40e2-b958-02891a86ea67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:23.333795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b9de0-6356-40e2-b958-02891a86ea67 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:23.345158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7712c4-162e-4244-aa24-707e5d6f756e service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:23.350295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 923/1860] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:35:23.352970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062d409-37ea-483b-8090-52958582ee6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:23.354139 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062d409-37ea-483b-8090-52958582ee6d service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:23.354537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062d409-37ea-483b-8090-52958582ee6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:23.355081 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062d409-37ea-483b-8090-52958582ee6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:23.355291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b9de0-6356-40e2-b958-02891a86ea67 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:23.357662 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f062d409-37ea-483b-8090-52958582ee6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:23.368724 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b9de0-6356-40e2-b958-02891a86ea67 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:23.373818 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a093ff3-f130-48b7-97a2-5849d22e93d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:23.375527 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a093ff3-f130-48b7-97a2-5849d22e93d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:23.376291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a093ff3-f130-48b7-97a2-5849d22e93d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:23.376291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a093ff3-f130-48b7-97a2-5849d22e93d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:23.376918 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a093ff3-f130-48b7-97a2-5849d22e93d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:23.379592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a093ff3-f130-48b7-97a2-5849d22e93d0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:23.390479 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:23.434815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 924/1861] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:23 2022] POST /identity/v3/projects => generated 367 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:23.436190 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a093ff3-f130-48b7-97a2-5849d22e93d0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:23.458278 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062d409-37ea-483b-8090-52958582ee6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:23.460111 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062d409-37ea-483b-8090-52958582ee6d service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:23.496743 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 938/1862] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:23.520945 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a093ff3-f130-48b7-97a2-5849d22e93d0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:23.526728 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743b44e-8050-4d69-a2ec-ae18fc8847e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:23.527843 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743b44e-8050-4d69-a2ec-ae18fc8847e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:23.528214 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743b44e-8050-4d69-a2ec-ae18fc8847e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:23.528579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743b44e-8050-4d69-a2ec-ae18fc8847e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:23.529390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8743b44e-8050-4d69-a2ec-ae18fc8847e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:23.531531 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8743b44e-8050-4d69-a2ec-ae18fc8847e4 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:23.561535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 925/1863] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:23 2022] POST /identity/v3/users => generated 284 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:23.563998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062d409-37ea-483b-8090-52958582ee6d service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:23.578743 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062d409-37ea-483b-8090-52958582ee6d service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:23.583960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2bd0d-bb2b-442a-b738-9a2c9bc20d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:23.584644 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2bd0d-bb2b-442a-b738-9a2c9bc20d60 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:23.584926 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2bd0d-bb2b-442a-b738-9a2c9bc20d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:23.585204 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2bd0d-bb2b-442a-b738-9a2c9bc20d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:23.586047 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2bd0d-bb2b-442a-b738-9a2c9bc20d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:23.587770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2bd0d-bb2b-442a-b738-9a2c9bc20d60 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:23.599661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 939/1864] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:23.603943 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743b44e-8050-4d69-a2ec-ae18fc8847e4 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:23.629375 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743b44e-8050-4d69-a2ec-ae18fc8847e4 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:23.643119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431df0a-36ac-4785-b30f-e704581e9dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:23.644247 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431df0a-36ac-4785-b30f-e704581e9dfb admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:23.644545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431df0a-36ac-4785-b30f-e704581e9dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:23.644937 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431df0a-36ac-4785-b30f-e704581e9dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:23.645752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c431df0a-36ac-4785-b30f-e704581e9dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:23.652328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c431df0a-36ac-4785-b30f-e704581e9dfb admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:23.668206 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 926/1865] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:23.670791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2bd0d-bb2b-442a-b738-9a2c9bc20d60 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:23.681762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2bd0d-bb2b-442a-b738-9a2c9bc20d60 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:23.689493 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f186c-163d-42c3-af1d-93b7757bb63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:23.690985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f186c-163d-42c3-af1d-93b7757bb63c admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:23.691275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f186c-163d-42c3-af1d-93b7757bb63c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:23.691551 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f186c-163d-42c3-af1d-93b7757bb63c admin admin] PATH_INFO: `/v3/projects/fd51cdfd74ec448582ffad5306d9b705/users/1941f3df12854d1da188bf32ada4ee25/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:23.722051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f186c-163d-42c3-af1d-93b7757bb63c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd51cdfd74ec448582ffad5306d9b705, user_id=1941f3df12854d1da188bf32ada4ee25, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:23.730055 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f186c-163d-42c3-af1d-93b7757bb63c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:23.758421 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 940/1866] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:23 2022] PUT /identity/v3/projects/fd51cdfd74ec448582ffad5306d9b705/users/1941f3df12854d1da188bf32ada4ee25/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:23.762588 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e3566-b623-46fc-9ea1-69583e665ea1 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:23.763109 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e3566-b623-46fc-9ea1-69583e665ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:23.763569 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e3566-b623-46fc-9ea1-69583e665ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:23.804048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-548e3566-b623-46fc-9ea1-69583e665ea1 None None] MFA Rules not processed for user `1941f3df12854d1da188bf32ada4ee25`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:23.805603 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-548e3566-b623-46fc-9ea1-69583e665ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:23.904365 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 927/1867] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:23 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:23.909160 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f186c-163d-42c3-af1d-93b7757bb63c admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:23.985330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:24.040045 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f186c-163d-42c3-af1d-93b7757bb63c admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:24.051758 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056f6f0-0c8a-4aca-99d6-4f5bfacdb313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:24.053151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056f6f0-0c8a-4aca-99d6-4f5bfacdb313 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:24.053440 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056f6f0-0c8a-4aca-99d6-4f5bfacdb313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:24.054028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056f6f0-0c8a-4aca-99d6-4f5bfacdb313 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:24.054750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056f6f0-0c8a-4aca-99d6-4f5bfacdb313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:24.061656 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056f6f0-0c8a-4aca-99d6-4f5bfacdb313 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:24.116557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 941/1868] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:23 2022] POST /identity/v3/projects => generated 365 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:24.120864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e3566-b623-46fc-9ea1-69583e665ea1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:24.167385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e3566-b623-46fc-9ea1-69583e665ea1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:24.227848 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:24.239599 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf679d-9b0e-4dda-be73-1d43c6cf11b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:24.240502 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf679d-9b0e-4dda-be73-1d43c6cf11b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:24.240879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf679d-9b0e-4dda-be73-1d43c6cf11b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:24.241611 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf679d-9b0e-4dda-be73-1d43c6cf11b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:24.242218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf679d-9b0e-4dda-be73-1d43c6cf11b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:24.244848 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf679d-9b0e-4dda-be73-1d43c6cf11b6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:24.289165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 928/1869] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:24 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:24.302082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056f6f0-0c8a-4aca-99d6-4f5bfacdb313 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:24.318223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056f6f0-0c8a-4aca-99d6-4f5bfacdb313 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:24.325238 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7be8f8-0701-4e8c-8d60-ac0b81a92719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:24.325952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7be8f8-0701-4e8c-8d60-ac0b81a92719 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:24.326239 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7be8f8-0701-4e8c-8d60-ac0b81a92719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:24.326518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7be8f8-0701-4e8c-8d60-ac0b81a92719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:24.327311 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7be8f8-0701-4e8c-8d60-ac0b81a92719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:24.329668 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7be8f8-0701-4e8c-8d60-ac0b81a92719 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:24.347725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 942/1870] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:24.351916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf679d-9b0e-4dda-be73-1d43c6cf11b6 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:24.371337 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf679d-9b0e-4dda-be73-1d43c6cf11b6 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:24.382369 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7788c-60b0-414a-ad7a-60c401c0e563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:24.383502 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7788c-60b0-414a-ad7a-60c401c0e563 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:24.383875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7788c-60b0-414a-ad7a-60c401c0e563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:24.384338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7788c-60b0-414a-ad7a-60c401c0e563 admin admin] PATH_INFO: `/v3/projects/9fe3d19ed3b84fb9b2db18c0652ccb3e/users/e112823d7b6f402d8cba2ddbd5799b71/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:24.409638 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7be8f8-0701-4e8c-8d60-ac0b81a92719 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:24.422664 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7be8f8-0701-4e8c-8d60-ac0b81a92719 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:24.424702 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7788c-60b0-414a-ad7a-60c401c0e563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fe3d19ed3b84fb9b2db18c0652ccb3e, user_id=e112823d7b6f402d8cba2ddbd5799b71, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:24.427471 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7788c-60b0-414a-ad7a-60c401c0e563 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:24.435994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4903d1a-37b5-4c72-9476-e29bc6f3992d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:24.437561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4903d1a-37b5-4c72-9476-e29bc6f3992d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:24.437561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4903d1a-37b5-4c72-9476-e29bc6f3992d admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:24.437561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4903d1a-37b5-4c72-9476-e29bc6f3992d admin admin] PATH_INFO: `/v3/users/a27750452bd34280a9102e093fc042c6` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:24.439030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4903d1a-37b5-4c72-9476-e29bc6f3992d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a27750452bd34280a9102e093fc042c6)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:24.441824 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4903d1a-37b5-4c72-9476-e29bc6f3992d admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:24.463426 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 929/1871] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:24 2022] PUT /identity/v3/projects/9fe3d19ed3b84fb9b2db18c0652ccb3e/users/e112823d7b6f402d8cba2ddbd5799b71/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:24.467447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7788c-60b0-414a-ad7a-60c401c0e563 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:24.547485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e4903d1a-37b5-4c72-9476-e29bc6f3992d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a27750452bd34280a9102e093fc042c6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:24.561517 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e4903d1a-37b5-4c72-9476-e29bc6f3992d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a27750452bd34280a9102e093fc042c6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:24.575224 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e4903d1a-37b5-4c72-9476-e29bc6f3992d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a27750452bd34280a9102e093fc042c6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:24.576138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e4903d1a-37b5-4c72-9476-e29bc6f3992d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a27750452bd34280a9102e093fc042c6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:24.576715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e4903d1a-37b5-4c72-9476-e29bc6f3992d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a27750452bd34280a9102e093fc042c6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:24.581902 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7788c-60b0-414a-ad7a-60c401c0e563 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:24.588516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0f96e-0ffc-45f0-8225-596453360a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:24.589351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0f96e-0ffc-45f0-8225-596453360a36 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:24.590308 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0f96e-0ffc-45f0-8225-596453360a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:24.590693 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0f96e-0ffc-45f0-8225-596453360a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:24.592758 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b0f96e-0ffc-45f0-8225-596453360a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:24.594215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b0f96e-0ffc-45f0-8225-596453360a36 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:24.607912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 943/1872] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:24 2022] DELETE /identity/v3/users/a27750452bd34280a9102e093fc042c6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:24.609409 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64cee5c-a7dc-4166-8b14-ef87769b0ada None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:24.609700 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64cee5c-a7dc-4166-8b14-ef87769b0ada None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:24.609977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64cee5c-a7dc-4166-8b14-ef87769b0ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:24.610970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 930/1873] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:24.615468 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0f96e-0ffc-45f0-8225-596453360a36 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:24.618272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-74b0f96e-0ffc-45f0-8225-596453360a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:24.635029 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0f96e-0ffc-45f0-8225-596453360a36 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:24.651430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-f64cee5c-a7dc-4166-8b14-ef87769b0ada None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:24.652654 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f64cee5c-a7dc-4166-8b14-ef87769b0ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:24.745983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d862ece-42a1-45b3-a96c-72ed20a5bc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:24.747725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d862ece-42a1-45b3-a96c-72ed20a5bc41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:24.749473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d862ece-42a1-45b3-a96c-72ed20a5bc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:24.749776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d862ece-42a1-45b3-a96c-72ed20a5bc41 admin admin] PATH_INFO: `/v3/projects/9fe3d19ed3b84fb9b2db18c0652ccb3e/users/e112823d7b6f402d8cba2ddbd5799b71/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:24.752600 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d862ece-42a1-45b3-a96c-72ed20a5bc41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fe3d19ed3b84fb9b2db18c0652ccb3e, user_id=e112823d7b6f402d8cba2ddbd5799b71, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:24.758815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d862ece-42a1-45b3-a96c-72ed20a5bc41 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:24.770889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 944/1874] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:24.785403 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 931/1875] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:24 2022] PUT /identity/v3/projects/9fe3d19ed3b84fb9b2db18c0652ccb3e/users/e112823d7b6f402d8cba2ddbd5799b71/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:24.796460 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d862ece-42a1-45b3-a96c-72ed20a5bc41 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:24.797210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952534f8-852c-4404-8ce4-0b09cfb8e03a None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:24.797604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952534f8-852c-4404-8ce4-0b09cfb8e03a None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:24.797787 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952534f8-852c-4404-8ce4-0b09cfb8e03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:24.800541 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8d862ece-42a1-45b3-a96c-72ed20a5bc41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:24.869427 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-952534f8-852c-4404-8ce4-0b09cfb8e03a None None] MFA Rules not processed for user `e112823d7b6f402d8cba2ddbd5799b71`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:24.870652 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-952534f8-852c-4404-8ce4-0b09cfb8e03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:24.880942 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:24.912216 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d862ece-42a1-45b3-a96c-72ed20a5bc41 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:24.917253 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820e6a2-ed89-4e73-834d-05eab55cfbaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:24.917546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820e6a2-ed89-4e73-834d-05eab55cfbaf service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:24.917824 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820e6a2-ed89-4e73-834d-05eab55cfbaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:24.918764 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820e6a2-ed89-4e73-834d-05eab55cfbaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:24.925307 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:24.979338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 945/1876] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:24 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:24.989933 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952534f8-852c-4404-8ce4-0b09cfb8e03a None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:24.992493 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-952534f8-852c-4404-8ce4-0b09cfb8e03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:25.014302 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e820e6a2-ed89-4e73-834d-05eab55cfbaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:25.017020 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e820e6a2-ed89-4e73-834d-05eab55cfbaf service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:25.062591 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 932/1877] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:25.081540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952534f8-852c-4404-8ce4-0b09cfb8e03a None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:25.086332 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2e931-578c-48fc-8c2f-7301a19adc4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:25.087057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2e931-578c-48fc-8c2f-7301a19adc4f service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:25.087344 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2e931-578c-48fc-8c2f-7301a19adc4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:25.087624 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2e931-578c-48fc-8c2f-7301a19adc4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:25.114179 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2e931-578c-48fc-8c2f-7301a19adc4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:25.116682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2e931-578c-48fc-8c2f-7301a19adc4f service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:25.148862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 946/1878] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:24 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:35:25.323553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820e6a2-ed89-4e73-834d-05eab55cfbaf service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:25.325234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e820e6a2-ed89-4e73-834d-05eab55cfbaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:25.342784 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820e6a2-ed89-4e73-834d-05eab55cfbaf service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:25.347496 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd458bd-5ce2-4b1b-a8ed-5cc44d09bb1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:25.348464 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd458bd-5ce2-4b1b-a8ed-5cc44d09bb1a service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:25.348610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd458bd-5ce2-4b1b-a8ed-5cc44d09bb1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:25.348890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd458bd-5ce2-4b1b-a8ed-5cc44d09bb1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:25.352119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1fd458bd-5ce2-4b1b-a8ed-5cc44d09bb1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:25.461508 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd458bd-5ce2-4b1b-a8ed-5cc44d09bb1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:25.463160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd458bd-5ce2-4b1b-a8ed-5cc44d09bb1a service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:25.481775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 933/1879] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:25 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:25.845545 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2e931-578c-48fc-8c2f-7301a19adc4f service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:25.846632 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e7f2e931-578c-48fc-8c2f-7301a19adc4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:25.904282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2e931-578c-48fc-8c2f-7301a19adc4f service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:25.930971 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f014623c-e7e3-47b4-8734-4ae2dec7a804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:25.931750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014623c-e7e3-47b4-8734-4ae2dec7a804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:25.932035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014623c-e7e3-47b4-8734-4ae2dec7a804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:25.932313 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014623c-e7e3-47b4-8734-4ae2dec7a804 admin admin] PATH_INFO: `/v3/projects/780591457f36403183bfb2828ef83e1f` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:25.940422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f014623c-e7e3-47b4-8734-4ae2dec7a804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=780591457f36403183bfb2828ef83e1f)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:25.942364 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f014623c-e7e3-47b4-8734-4ae2dec7a804 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:26.066856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f014623c-e7e3-47b4-8734-4ae2dec7a804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '780591457f36403183bfb2828ef83e1f', 'request_id': 'req-f014623c-e7e3-47b4-8734-4ae2dec7a804'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:26.079863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f014623c-e7e3-47b4-8734-4ae2dec7a804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '780591457f36403183bfb2828ef83e1f', 'request_id': 'req-f014623c-e7e3-47b4-8734-4ae2dec7a804'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:26.091414 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f014623c-e7e3-47b4-8734-4ae2dec7a804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '780591457f36403183bfb2828ef83e1f', 'request_id': 'req-f014623c-e7e3-47b4-8734-4ae2dec7a804'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:26.103835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f014623c-e7e3-47b4-8734-4ae2dec7a804 admin admin] The token cache is being invalidate because project 780591457f36403183bfb2828ef83e1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:26.104521 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f014623c-e7e3-47b4-8734-4ae2dec7a804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:26.105404 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f014623c-e7e3-47b4-8734-4ae2dec7a804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:26.107867 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 947/1880] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:25 2022] DELETE /identity/v3/projects/780591457f36403183bfb2828ef83e1f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:26.112192 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd458bd-5ce2-4b1b-a8ed-5cc44d09bb1a service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:26.131066 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd458bd-5ce2-4b1b-a8ed-5cc44d09bb1a service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:26.216366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972073d2-6b60-47b2-9841-19efe3e3df04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:26.217331 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972073d2-6b60-47b2-9841-19efe3e3df04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:26.217641 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972073d2-6b60-47b2-9841-19efe3e3df04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:26.217935 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972073d2-6b60-47b2-9841-19efe3e3df04 admin admin] PATH_INFO: `/v3/users/09b04922eabc4b359eeae2554e81bbd0` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:26.219475 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972073d2-6b60-47b2-9841-19efe3e3df04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b04922eabc4b359eeae2554e81bbd0)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:26.222438 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972073d2-6b60-47b2-9841-19efe3e3df04 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:26.310078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:26.311897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-972073d2-6b60-47b2-9841-19efe3e3df04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b04922eabc4b359eeae2554e81bbd0'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:26.322258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-972073d2-6b60-47b2-9841-19efe3e3df04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b04922eabc4b359eeae2554e81bbd0'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:26.323139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-972073d2-6b60-47b2-9841-19efe3e3df04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b04922eabc4b359eeae2554e81bbd0'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:26.324730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-972073d2-6b60-47b2-9841-19efe3e3df04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b04922eabc4b359eeae2554e81bbd0'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:26.340165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-972073d2-6b60-47b2-9841-19efe3e3df04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b04922eabc4b359eeae2554e81bbd0'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:26.353894 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 934/1881] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:26 2022] DELETE /identity/v3/users/09b04922eabc4b359eeae2554e81bbd0 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:26.652697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f014623c-e7e3-47b4-8734-4ae2dec7a804 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:26.654116 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f014623c-e7e3-47b4-8734-4ae2dec7a804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:26.672103 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f014623c-e7e3-47b4-8734-4ae2dec7a804 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:26.706320 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:26.737503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74bce0-a6d6-46de-8b41-6e5852273ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:26.738852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74bce0-a6d6-46de-8b41-6e5852273ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:26.739399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74bce0-a6d6-46de-8b41-6e5852273ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:26.739897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74bce0-a6d6-46de-8b41-6e5852273ffc admin admin] PATH_INFO: `/v3/projects/8349d432c68e4fbaab8a3a7fb5b9aec1` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:26.741314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74bce0-a6d6-46de-8b41-6e5852273ffc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8349d432c68e4fbaab8a3a7fb5b9aec1)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:26.743449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74bce0-a6d6-46de-8b41-6e5852273ffc admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:26.793322 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-aa74bce0-a6d6-46de-8b41-6e5852273ffc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8349d432c68e4fbaab8a3a7fb5b9aec1', 'request_id': 'req-aa74bce0-a6d6-46de-8b41-6e5852273ffc'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:26.800951 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-aa74bce0-a6d6-46de-8b41-6e5852273ffc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8349d432c68e4fbaab8a3a7fb5b9aec1', 'request_id': 'req-aa74bce0-a6d6-46de-8b41-6e5852273ffc'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:26.807860 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-aa74bce0-a6d6-46de-8b41-6e5852273ffc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8349d432c68e4fbaab8a3a7fb5b9aec1', 'request_id': 'req-aa74bce0-a6d6-46de-8b41-6e5852273ffc'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:26.821255 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-aa74bce0-a6d6-46de-8b41-6e5852273ffc admin admin] The token cache is being invalidate because project 8349d432c68e4fbaab8a3a7fb5b9aec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:26.821557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-aa74bce0-a6d6-46de-8b41-6e5852273ffc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:26.822359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-aa74bce0-a6d6-46de-8b41-6e5852273ffc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:26.829412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 948/1882] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:26 2022] DELETE /identity/v3/projects/8349d432c68e4fbaab8a3a7fb5b9aec1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:26.831774 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8845ec-791a-442c-bd7a-012cacb96ffb None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:26.832065 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8845ec-791a-442c-bd7a-012cacb96ffb None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:26.832347 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8845ec-791a-442c-bd7a-012cacb96ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:26.853306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-fd8845ec-791a-442c-bd7a-012cacb96ffb None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:26.854496 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fd8845ec-791a-442c-bd7a-012cacb96ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:26.920911 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 935/1883] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:26.928766 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a9ad4-62ad-48b6-8f56-1a7632e8a622 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:26.929270 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a9ad4-62ad-48b6-8f56-1a7632e8a622 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:26.929602 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a9ad4-62ad-48b6-8f56-1a7632e8a622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:26.952852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-bc4a9ad4-62ad-48b6-8f56-1a7632e8a622 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:26.954893 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-bc4a9ad4-62ad-48b6-8f56-1a7632e8a622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:26.967000 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 949/1884] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:26.977690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8845ec-791a-442c-bd7a-012cacb96ffb None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:26.988600 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8845ec-791a-442c-bd7a-012cacb96ffb None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:26.994170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4335c2c-3145-432c-bfaa-ced8b2510373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:26.995072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4335c2c-3145-432c-bfaa-ced8b2510373 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:26.995462 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4335c2c-3145-432c-bfaa-ced8b2510373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:26.995845 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4335c2c-3145-432c-bfaa-ced8b2510373 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:26.996670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4335c2c-3145-432c-bfaa-ced8b2510373 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:35:26.997323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4335c2c-3145-432c-bfaa-ced8b2510373 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:26.998754 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4335c2c-3145-432c-bfaa-ced8b2510373 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:27.016417 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 936/1885] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:35:27.017333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a9ad4-62ad-48b6-8f56-1a7632e8a622 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:27.029620 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a9ad4-62ad-48b6-8f56-1a7632e8a622 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:27.035862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f643ed8-c874-4983-9db2-88eaa49ea64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:27.036647 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f643ed8-c874-4983-9db2-88eaa49ea64c admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:27.037038 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f643ed8-c874-4983-9db2-88eaa49ea64c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:27.037717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f643ed8-c874-4983-9db2-88eaa49ea64c admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:27.045781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f643ed8-c874-4983-9db2-88eaa49ea64c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:27.047757 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f643ed8-c874-4983-9db2-88eaa49ea64c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:27.085820 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 950/1886] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:27 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:27.090664 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4335c2c-3145-432c-bfaa-ced8b2510373 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:27.105225 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4335c2c-3145-432c-bfaa-ced8b2510373 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:27.184103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fe9da-2398-43e4-b9ed-68da085abc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:27.185010 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fe9da-2398-43e4-b9ed-68da085abc04 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:27.185393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fe9da-2398-43e4-b9ed-68da085abc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:27.185752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fe9da-2398-43e4-b9ed-68da085abc04 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:27.186537 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fe9da-2398-43e4-b9ed-68da085abc04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:27.188732 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fe9da-2398-43e4-b9ed-68da085abc04 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:27.232405 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 937/1887] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:27 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:35:27.232885 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f643ed8-c874-4983-9db2-88eaa49ea64c admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:27.248727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f643ed8-c874-4983-9db2-88eaa49ea64c admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:27.256801 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd894a-12f7-491e-902d-4be0fe188b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:27.258561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd894a-12f7-491e-902d-4be0fe188b4d admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:27.258848 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd894a-12f7-491e-902d-4be0fe188b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:27.259192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd894a-12f7-491e-902d-4be0fe188b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:27.259912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd894a-12f7-491e-902d-4be0fe188b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:27.262303 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd894a-12f7-491e-902d-4be0fe188b4d admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:27.285693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 951/1888] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:27.289611 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fe9da-2398-43e4-b9ed-68da085abc04 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:27.307148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fe9da-2398-43e4-b9ed-68da085abc04 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:27.317224 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91934711-6f12-4b8d-b981-a9d79fa7abd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:27.318063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91934711-6f12-4b8d-b981-a9d79fa7abd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:27.318446 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91934711-6f12-4b8d-b981-a9d79fa7abd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:27.319048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91934711-6f12-4b8d-b981-a9d79fa7abd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:27.320090 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91934711-6f12-4b8d-b981-a9d79fa7abd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:27.322028 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91934711-6f12-4b8d-b981-a9d79fa7abd0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:27.334921 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 938/1889] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:27.338929 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd894a-12f7-491e-902d-4be0fe188b4d admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:27.351150 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd894a-12f7-491e-902d-4be0fe188b4d admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:27.357093 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42268f-b770-4524-bef2-eb41ae6d57b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:27.357912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42268f-b770-4524-bef2-eb41ae6d57b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:27.358197 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42268f-b770-4524-bef2-eb41ae6d57b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:27.358476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42268f-b770-4524-bef2-eb41ae6d57b7 admin admin] PATH_INFO: `/v3/projects/480c617c65b641c8bc0143ee0a8ec220/users/d845e53dbc66437793dcd4e140187012/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:27.395145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42268f-b770-4524-bef2-eb41ae6d57b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=480c617c65b641c8bc0143ee0a8ec220, user_id=d845e53dbc66437793dcd4e140187012, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:27.398307 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42268f-b770-4524-bef2-eb41ae6d57b7 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:27.413611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 952/1890] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:27 2022] PUT /identity/v3/projects/480c617c65b641c8bc0143ee0a8ec220/users/d845e53dbc66437793dcd4e140187012/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:35:27.425629 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55343077-d898-4bff-a71d-a6245e4e845e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:27.425971 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55343077-d898-4bff-a71d-a6245e4e845e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:27.426504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55343077-d898-4bff-a71d-a6245e4e845e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:27.463005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-55343077-d898-4bff-a71d-a6245e4e845e None None] MFA Rules not processed for user `d845e53dbc66437793dcd4e140187012`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:27.464307 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-55343077-d898-4bff-a71d-a6245e4e845e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:27.477200 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:27.553456 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 939/1891] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:27.564059 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42268f-b770-4524-bef2-eb41ae6d57b7 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:27.571714 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-5f42268f-b770-4524-bef2-eb41ae6d57b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:27.685726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:27.709369 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42268f-b770-4524-bef2-eb41ae6d57b7 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:27.717829 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9775c8e-705a-411d-9e6b-998d045d8986 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b7a38b4a7754037a00d11dafc88645b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:27.719222 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9775c8e-705a-411d-9e6b-998d045d8986 service glance] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:27.719874 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9775c8e-705a-411d-9e6b-998d045d8986 service glance] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:27.720501 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9775c8e-705a-411d-9e6b-998d045d8986 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:27.733861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9775c8e-705a-411d-9e6b-998d045d8986 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:27.735809 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9775c8e-705a-411d-9e6b-998d045d8986 service glance] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:27.746847 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 953/1892] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:27.827298 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55343077-d898-4bff-a71d-a6245e4e845e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:27.829183 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-55343077-d898-4bff-a71d-a6245e4e845e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:27.967931 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55343077-d898-4bff-a71d-a6245e4e845e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:27.979352 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe94db9-3aa4-4803-9914-f45e584571d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:27.980255 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe94db9-3aa4-4803-9914-f45e584571d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:27.980541 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe94db9-3aa4-4803-9914-f45e584571d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:27.980814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe94db9-3aa4-4803-9914-f45e584571d6 admin admin] PATH_INFO: `/v3/users/2e6d98a7fe104f90a8253cd37a932e79` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:27.984115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe94db9-3aa4-4803-9914-f45e584571d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e6d98a7fe104f90a8253cd37a932e79)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:27.987475 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe94db9-3aa4-4803-9914-f45e584571d6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:28.065170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9fe94db9-3aa4-4803-9914-f45e584571d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e6d98a7fe104f90a8253cd37a932e79'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:28.079881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9fe94db9-3aa4-4803-9914-f45e584571d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e6d98a7fe104f90a8253cd37a932e79'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:28.083280 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9fe94db9-3aa4-4803-9914-f45e584571d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e6d98a7fe104f90a8253cd37a932e79'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:28.083951 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9fe94db9-3aa4-4803-9914-f45e584571d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e6d98a7fe104f90a8253cd37a932e79'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:28.101030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9fe94db9-3aa4-4803-9914-f45e584571d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e6d98a7fe104f90a8253cd37a932e79'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:28.119026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 940/1893] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:27 2022] DELETE /identity/v3/users/2e6d98a7fe104f90a8253cd37a932e79 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:28.123912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1f0b9-143a-48bc-bd9c-67cec3a25b67 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:28.125417 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1f0b9-143a-48bc-bd9c-67cec3a25b67 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:28.127315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1f0b9-143a-48bc-bd9c-67cec3a25b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:28.155720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-35b1f0b9-143a-48bc-bd9c-67cec3a25b67 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:28.159330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-35b1f0b9-143a-48bc-bd9c-67cec3a25b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:28.257047 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 954/1894] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:35:28.270572 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe94db9-3aa4-4803-9914-f45e584571d6 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:28.272953 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9fe94db9-3aa4-4803-9914-f45e584571d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:28.340311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:28.384766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe94db9-3aa4-4803-9914-f45e584571d6 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:28.397128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb6b9b-ba4d-454b-a8ed-6ef578775b28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:28.397899 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb6b9b-ba4d-454b-a8ed-6ef578775b28 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:28.398344 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb6b9b-ba4d-454b-a8ed-6ef578775b28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:28.398631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb6b9b-ba4d-454b-a8ed-6ef578775b28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:28.411417 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb6b9b-ba4d-454b-a8ed-6ef578775b28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:28.414221 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb6b9b-ba4d-454b-a8ed-6ef578775b28 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:28.432210 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 941/1895] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:35:28.950753 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1f0b9-143a-48bc-bd9c-67cec3a25b67 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:28.952460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-35b1f0b9-143a-48bc-bd9c-67cec3a25b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:28.966697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1f0b9-143a-48bc-bd9c-67cec3a25b67 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:28.972094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b7f94-c222-454f-b971-d91c7837e876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:28.973051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b7f94-c222-454f-b971-d91c7837e876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:28.973333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b7f94-c222-454f-b971-d91c7837e876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:28.973605 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b7f94-c222-454f-b971-d91c7837e876 admin admin] PATH_INFO: `/v3/projects/754784e6d35a444481149ea200cdd961` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:28.975098 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b7f94-c222-454f-b971-d91c7837e876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=754784e6d35a444481149ea200cdd961)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:28.976960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b7f94-c222-454f-b971-d91c7837e876 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:29.057037 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-af8b7f94-c222-454f-b971-d91c7837e876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '754784e6d35a444481149ea200cdd961', 'request_id': 'req-af8b7f94-c222-454f-b971-d91c7837e876'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:29.065074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-af8b7f94-c222-454f-b971-d91c7837e876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '754784e6d35a444481149ea200cdd961', 'request_id': 'req-af8b7f94-c222-454f-b971-d91c7837e876'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:29.074202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-af8b7f94-c222-454f-b971-d91c7837e876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '754784e6d35a444481149ea200cdd961', 'request_id': 'req-af8b7f94-c222-454f-b971-d91c7837e876'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:29.087072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-af8b7f94-c222-454f-b971-d91c7837e876 admin admin] The token cache is being invalidate because project 754784e6d35a444481149ea200cdd961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:29.087369 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-af8b7f94-c222-454f-b971-d91c7837e876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:29.088036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-af8b7f94-c222-454f-b971-d91c7837e876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:29.089927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 955/1896] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:28 2022] DELETE /identity/v3/projects/754784e6d35a444481149ea200cdd961 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:29.093988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb6b9b-ba4d-454b-a8ed-6ef578775b28 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:29.122878 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb6b9b-ba4d-454b-a8ed-6ef578775b28 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:29.203888 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eba3dd-1bf8-4fa4-9e88-e703ecd10af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:29.205165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eba3dd-1bf8-4fa4-9e88-e703ecd10af8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:29.205447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eba3dd-1bf8-4fa4-9e88-e703ecd10af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:29.205726 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eba3dd-1bf8-4fa4-9e88-e703ecd10af8 admin admin] PATH_INFO: `/v3/users/e43168af115f42fd8a5938ce23353b39` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:29.212213 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eba3dd-1bf8-4fa4-9e88-e703ecd10af8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e43168af115f42fd8a5938ce23353b39)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:29.214191 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eba3dd-1bf8-4fa4-9e88-e703ecd10af8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:29.281562 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-75eba3dd-1bf8-4fa4-9e88-e703ecd10af8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e43168af115f42fd8a5938ce23353b39'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:29.293541 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-75eba3dd-1bf8-4fa4-9e88-e703ecd10af8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e43168af115f42fd8a5938ce23353b39'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:29.294375 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-75eba3dd-1bf8-4fa4-9e88-e703ecd10af8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e43168af115f42fd8a5938ce23353b39'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:29.295356 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-75eba3dd-1bf8-4fa4-9e88-e703ecd10af8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e43168af115f42fd8a5938ce23353b39'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:29.309528 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-75eba3dd-1bf8-4fa4-9e88-e703ecd10af8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e43168af115f42fd8a5938ce23353b39'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:29.330706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 942/1897] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:29 2022] DELETE /identity/v3/users/e43168af115f42fd8a5938ce23353b39 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:35:29.922672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b7f94-c222-454f-b971-d91c7837e876 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:29.924290 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-af8b7f94-c222-454f-b971-d91c7837e876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:29.949167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b7f94-c222-454f-b971-d91c7837e876 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:30.060711 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d567989-2581-4c34-9281-2ff5328201f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:30.061624 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d567989-2581-4c34-9281-2ff5328201f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:30.062040 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d567989-2581-4c34-9281-2ff5328201f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:30.062325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d567989-2581-4c34-9281-2ff5328201f6 admin admin] PATH_INFO: `/v3/projects/40c468289eb244ecae654bebe0b77034` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:30.063911 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d567989-2581-4c34-9281-2ff5328201f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40c468289eb244ecae654bebe0b77034)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:30.066369 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d567989-2581-4c34-9281-2ff5328201f6 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:30.077832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:30.130316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8d567989-2581-4c34-9281-2ff5328201f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40c468289eb244ecae654bebe0b77034', 'request_id': 'req-8d567989-2581-4c34-9281-2ff5328201f6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:30.146086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8d567989-2581-4c34-9281-2ff5328201f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40c468289eb244ecae654bebe0b77034', 'request_id': 'req-8d567989-2581-4c34-9281-2ff5328201f6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:30.152584 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8d567989-2581-4c34-9281-2ff5328201f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40c468289eb244ecae654bebe0b77034', 'request_id': 'req-8d567989-2581-4c34-9281-2ff5328201f6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:30.163411 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8d567989-2581-4c34-9281-2ff5328201f6 admin admin] The token cache is being invalidate because project 40c468289eb244ecae654bebe0b77034 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:30.163703 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8d567989-2581-4c34-9281-2ff5328201f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:30.164363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8d567989-2581-4c34-9281-2ff5328201f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:30.165916 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 956/1898] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:29 2022] DELETE /identity/v3/projects/40c468289eb244ecae654bebe0b77034 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:30.171098 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b586b18-c994-40d3-a8ca-a51f6c761d36 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:30.171614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b586b18-c994-40d3-a8ca-a51f6c761d36 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:30.171703 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b586b18-c994-40d3-a8ca-a51f6c761d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:30.191715 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-7b586b18-c994-40d3-a8ca-a51f6c761d36 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:30.192940 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7b586b18-c994-40d3-a8ca-a51f6c761d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:30.259929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:30.274348 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 943/1899] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:35:30.282197 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b50fb-6094-4ad7-bf32-9eb3a9d3ba2b None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:30.282489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b50fb-6094-4ad7-bf32-9eb3a9d3ba2b None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:30.282772 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b50fb-6094-4ad7-bf32-9eb3a9d3ba2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:30.302178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-357b50fb-6094-4ad7-bf32-9eb3a9d3ba2b None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:30.303419 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-357b50fb-6094-4ad7-bf32-9eb3a9d3ba2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:30.312638 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 957/1900] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:30.317106 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b586b18-c994-40d3-a8ca-a51f6c761d36 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:30.326965 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b586b18-c994-40d3-a8ca-a51f6c761d36 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:30.331944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610da524-299c-4255-aed8-8823729ed552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:30.332605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610da524-299c-4255-aed8-8823729ed552 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:30.332888 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610da524-299c-4255-aed8-8823729ed552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:30.333159 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610da524-299c-4255-aed8-8823729ed552 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:30.333880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610da524-299c-4255-aed8-8823729ed552 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:35:30.334183 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610da524-299c-4255-aed8-8823729ed552 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:30.335814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610da524-299c-4255-aed8-8823729ed552 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:30.349404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 944/1901] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:35:30.354099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b50fb-6094-4ad7-bf32-9eb3a9d3ba2b None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:30.366087 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b50fb-6094-4ad7-bf32-9eb3a9d3ba2b None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:30.371355 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d01721-0736-4973-aa6f-18ab323b8194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:30.372034 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d01721-0736-4973-aa6f-18ab323b8194 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:30.372317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d01721-0736-4973-aa6f-18ab323b8194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:30.372907 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d01721-0736-4973-aa6f-18ab323b8194 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:30.373728 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d01721-0736-4973-aa6f-18ab323b8194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:30.375282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d01721-0736-4973-aa6f-18ab323b8194 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:30.408421 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 958/1902] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:30 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:30.409647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610da524-299c-4255-aed8-8823729ed552 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:30.421282 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610da524-299c-4255-aed8-8823729ed552 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:30.488310 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91a10f-a89c-466a-9daa-62857b463f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:30.488988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91a10f-a89c-466a-9daa-62857b463f54 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:30.489276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91a10f-a89c-466a-9daa-62857b463f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:30.489619 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91a10f-a89c-466a-9daa-62857b463f54 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:30.490251 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91a10f-a89c-466a-9daa-62857b463f54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:30.492200 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91a10f-a89c-466a-9daa-62857b463f54 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:30.512092 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 945/1903] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:30 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:30.516174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d01721-0736-4973-aa6f-18ab323b8194 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:30.527002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d01721-0736-4973-aa6f-18ab323b8194 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:30.531491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff830b-5289-4e84-8c95-5a93a706ebd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:30.532150 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff830b-5289-4e84-8c95-5a93a706ebd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:30.532429 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff830b-5289-4e84-8c95-5a93a706ebd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:30.532703 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff830b-5289-4e84-8c95-5a93a706ebd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:30.534039 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff830b-5289-4e84-8c95-5a93a706ebd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:30.535685 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff830b-5289-4e84-8c95-5a93a706ebd1 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:30.546190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 959/1904] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:30.551007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91a10f-a89c-466a-9daa-62857b463f54 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:30.561750 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91a10f-a89c-466a-9daa-62857b463f54 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:30.565751 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4dc91f-25c3-481c-a3db-7aed2448fed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:30.566441 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dc91f-25c3-481c-a3db-7aed2448fed3 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:30.566721 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dc91f-25c3-481c-a3db-7aed2448fed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:30.567021 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dc91f-25c3-481c-a3db-7aed2448fed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:30.567756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4dc91f-25c3-481c-a3db-7aed2448fed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:30.569335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4dc91f-25c3-481c-a3db-7aed2448fed3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:30.581320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 946/1905] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:35:30.583967 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff830b-5289-4e84-8c95-5a93a706ebd1 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:30.594312 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff830b-5289-4e84-8c95-5a93a706ebd1 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:30.599504 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c9b06-765a-4674-bf33-10a280c7e60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:30.600430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c9b06-765a-4674-bf33-10a280c7e60b admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:30.600715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c9b06-765a-4674-bf33-10a280c7e60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:30.600995 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c9b06-765a-4674-bf33-10a280c7e60b admin admin] PATH_INFO: `/v3/projects/55ec62064c8d4380996ab0a5de9c0b38/users/09e00efecab742d78df7f9aa6ae81a85/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:30.616563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c9b06-765a-4674-bf33-10a280c7e60b admin admin] RBAC: Authorizing `identity:create_grant(project_id=55ec62064c8d4380996ab0a5de9c0b38, user_id=09e00efecab742d78df7f9aa6ae81a85, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:30.619255 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c9b06-765a-4674-bf33-10a280c7e60b admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:30.641845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 960/1906] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:30 2022] PUT /identity/v3/projects/55ec62064c8d4380996ab0a5de9c0b38/users/09e00efecab742d78df7f9aa6ae81a85/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:30.645053 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271af5d-2b8d-4631-b5ba-fc117d63dc1b None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:30.645343 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271af5d-2b8d-4631-b5ba-fc117d63dc1b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:30.645624 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271af5d-2b8d-4631-b5ba-fc117d63dc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:30.674602 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e271af5d-2b8d-4631-b5ba-fc117d63dc1b None None] MFA Rules not processed for user `09e00efecab742d78df7f9aa6ae81a85`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:30.675870 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e271af5d-2b8d-4631-b5ba-fc117d63dc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:30.743052 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 947/1907] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:30.749491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c9b06-765a-4674-bf33-10a280c7e60b admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:30.812185 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c9b06-765a-4674-bf33-10a280c7e60b admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:30.816771 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2fefb-b851-4004-a373-3d59bad4c9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:30.817433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2fefb-b851-4004-a373-3d59bad4c9ed admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:30.817721 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2fefb-b851-4004-a373-3d59bad4c9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:30.817999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2fefb-b851-4004-a373-3d59bad4c9ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:30.818689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2fefb-b851-4004-a373-3d59bad4c9ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:30.820618 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2fefb-b851-4004-a373-3d59bad4c9ed admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:30.846049 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 961/1908] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:30 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:30.850351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271af5d-2b8d-4631-b5ba-fc117d63dc1b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:30.862089 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271af5d-2b8d-4631-b5ba-fc117d63dc1b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:30.909553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:30.921131 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57abe8-3cad-4d28-9d26-f952edffb851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:30.921814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57abe8-3cad-4d28-9d26-f952edffb851 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:30.922095 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57abe8-3cad-4d28-9d26-f952edffb851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:30.922371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57abe8-3cad-4d28-9d26-f952edffb851 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:30.923112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57abe8-3cad-4d28-9d26-f952edffb851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:30.925075 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57abe8-3cad-4d28-9d26-f952edffb851 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:30.944732 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 948/1909] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:30 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:30.948815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2fefb-b851-4004-a373-3d59bad4c9ed admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:30.959793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2fefb-b851-4004-a373-3d59bad4c9ed admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:30.964378 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba13c2-8857-4749-93e2-f88c4f255521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:30.965028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba13c2-8857-4749-93e2-f88c4f255521 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:30.965309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba13c2-8857-4749-93e2-f88c4f255521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:30.965603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba13c2-8857-4749-93e2-f88c4f255521 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:30.966307 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba13c2-8857-4749-93e2-f88c4f255521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:30.967949 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba13c2-8857-4749-93e2-f88c4f255521 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:30.979399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 962/1910] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:30.983413 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57abe8-3cad-4d28-9d26-f952edffb851 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:30.993361 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57abe8-3cad-4d28-9d26-f952edffb851 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:30.997936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7e372-9e91-442a-a28f-fc41f79ce656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:30.998596 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7e372-9e91-442a-a28f-fc41f79ce656 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:30.998883 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7e372-9e91-442a-a28f-fc41f79ce656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:30.999192 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7e372-9e91-442a-a28f-fc41f79ce656 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:30.999917 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7e372-9e91-442a-a28f-fc41f79ce656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:31.001513 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7e372-9e91-442a-a28f-fc41f79ce656 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:31.011446 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 949/1911] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:35:31.015559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba13c2-8857-4749-93e2-f88c4f255521 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:31.032057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba13c2-8857-4749-93e2-f88c4f255521 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:31.037445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883afe0-a0c6-4011-a5a5-dc2b9f3dc729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:31.038335 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883afe0-a0c6-4011-a5a5-dc2b9f3dc729 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:31.038619 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883afe0-a0c6-4011-a5a5-dc2b9f3dc729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:31.038897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883afe0-a0c6-4011-a5a5-dc2b9f3dc729 admin admin] PATH_INFO: `/v3/projects/6a48e3d9d3c54088b99685ce39d9910e/users/dbb61a7f38394fc298cabf78b499106e/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:31.041282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:31.064578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8883afe0-a0c6-4011-a5a5-dc2b9f3dc729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a48e3d9d3c54088b99685ce39d9910e, user_id=dbb61a7f38394fc298cabf78b499106e, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:31.067288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8883afe0-a0c6-4011-a5a5-dc2b9f3dc729 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:31.102532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 963/1912] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:31 2022] PUT /identity/v3/projects/6a48e3d9d3c54088b99685ce39d9910e/users/dbb61a7f38394fc298cabf78b499106e/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:31.116818 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91963ed8-cc27-4d71-be85-af663e64e356 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:31.117110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91963ed8-cc27-4d71-be85-af663e64e356 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:31.117390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91963ed8-cc27-4d71-be85-af663e64e356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:31.146982 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-91963ed8-cc27-4d71-be85-af663e64e356 None None] MFA Rules not processed for user `dbb61a7f38394fc298cabf78b499106e`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:31.148362 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-91963ed8-cc27-4d71-be85-af663e64e356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:31.251554 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 950/1913] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:31.256009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883afe0-a0c6-4011-a5a5-dc2b9f3dc729 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:31.288035 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91963ed8-cc27-4d71-be85-af663e64e356 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:31.289562 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-91963ed8-cc27-4d71-be85-af663e64e356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:31.334803 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883afe0-a0c6-4011-a5a5-dc2b9f3dc729 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:31.341719 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530714f-3522-4d8e-a685-5543ae6ad4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:31.342409 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530714f-3522-4d8e-a685-5543ae6ad4f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:31.342737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530714f-3522-4d8e-a685-5543ae6ad4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:31.343048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530714f-3522-4d8e-a685-5543ae6ad4f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:31.343759 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530714f-3522-4d8e-a685-5543ae6ad4f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:31.345707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530714f-3522-4d8e-a685-5543ae6ad4f0 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:31.374604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91963ed8-cc27-4d71-be85-af663e64e356 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:31.379049 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 964/1914] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:31 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:31.380617 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530714f-3522-4d8e-a685-5543ae6ad4f0 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:31.446040 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e781b-c89c-41aa-a2d7-42fb23daf91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:31.447004 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e781b-c89c-41aa-a2d7-42fb23daf91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:31.447292 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e781b-c89c-41aa-a2d7-42fb23daf91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:31.447778 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e781b-c89c-41aa-a2d7-42fb23daf91e admin admin] PATH_INFO: `/v3/users/1941f3df12854d1da188bf32ada4ee25` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:31.448779 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e781b-c89c-41aa-a2d7-42fb23daf91e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1941f3df12854d1da188bf32ada4ee25)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:31.450739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e781b-c89c-41aa-a2d7-42fb23daf91e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:31.461489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530714f-3522-4d8e-a685-5543ae6ad4f0 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:31.463228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:31.468756 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80577387-fde5-4f69-87bb-645c3e2b82fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:31.470439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80577387-fde5-4f69-87bb-645c3e2b82fd admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:31.470913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80577387-fde5-4f69-87bb-645c3e2b82fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:31.471300 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80577387-fde5-4f69-87bb-645c3e2b82fd admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:31.472127 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80577387-fde5-4f69-87bb-645c3e2b82fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:31.474082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80577387-fde5-4f69-87bb-645c3e2b82fd admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:31.507051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 965/1915] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:31 2022] POST /identity/v3/users => generated 295 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:31.511584 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80577387-fde5-4f69-87bb-645c3e2b82fd admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:31.522759 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80577387-fde5-4f69-87bb-645c3e2b82fd admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:31.528416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aa57a-4804-4dd9-aebd-ab7166fecaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:31.528699 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aa57a-4804-4dd9-aebd-ab7166fecaf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:31.528994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aa57a-4804-4dd9-aebd-ab7166fecaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:31.529273 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aa57a-4804-4dd9-aebd-ab7166fecaf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:31.530029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503aa57a-4804-4dd9-aebd-ab7166fecaf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:31.531668 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503aa57a-4804-4dd9-aebd-ab7166fecaf7 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:31.536941 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a35e781b-c89c-41aa-a2d7-42fb23daf91e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1941f3df12854d1da188bf32ada4ee25'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:31.542978 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 966/1916] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:35:31.546036 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a35e781b-c89c-41aa-a2d7-42fb23daf91e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1941f3df12854d1da188bf32ada4ee25'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:31.546832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a35e781b-c89c-41aa-a2d7-42fb23daf91e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1941f3df12854d1da188bf32ada4ee25'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:31.547423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aa57a-4804-4dd9-aebd-ab7166fecaf7 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:31.547532 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a35e781b-c89c-41aa-a2d7-42fb23daf91e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1941f3df12854d1da188bf32ada4ee25'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:31.549426 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-503aa57a-4804-4dd9-aebd-ab7166fecaf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:31.562984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aa57a-4804-4dd9-aebd-ab7166fecaf7 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:31.568169 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a35e781b-c89c-41aa-a2d7-42fb23daf91e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1941f3df12854d1da188bf32ada4ee25'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:31.572701 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caeb99f-4409-417d-a28f-f02486391a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:31.573604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caeb99f-4409-417d-a28f-f02486391a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:31.574166 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caeb99f-4409-417d-a28f-f02486391a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:31.574449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caeb99f-4409-417d-a28f-f02486391a66 admin admin] PATH_INFO: `/v3/projects/f41b81967c824448ba06e5aecd60f3e0/users/1bfa1ee424574a60b11dff8934353329/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:31.591813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caeb99f-4409-417d-a28f-f02486391a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f41b81967c824448ba06e5aecd60f3e0, user_id=1bfa1ee424574a60b11dff8934353329, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:31.592897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 951/1917] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:31 2022] DELETE /identity/v3/users/1941f3df12854d1da188bf32ada4ee25 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:35:31.593693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caeb99f-4409-417d-a28f-f02486391a66 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:31.599552 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe300e-8174-46e3-b3e9-99793e11059b None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:31.600389 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe300e-8174-46e3-b3e9-99793e11059b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:31.600865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe300e-8174-46e3-b3e9-99793e11059b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:31.609290 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 967/1918] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:31 2022] PUT /identity/v3/projects/f41b81967c824448ba06e5aecd60f3e0/users/1bfa1ee424574a60b11dff8934353329/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:35:31.613196 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caeb99f-4409-417d-a28f-f02486391a66 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:31.625847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1dfe300e-8174-46e3-b3e9-99793e11059b None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:31.630442 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1dfe300e-8174-46e3-b3e9-99793e11059b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:31.648570 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:31.695049 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caeb99f-4409-417d-a28f-f02486391a66 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:31.700524 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219aada5-606a-4e77-9b01-693f237b3e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:31.701052 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219aada5-606a-4e77-9b01-693f237b3e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:31.701314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219aada5-606a-4e77-9b01-693f237b3e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:31.701582 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219aada5-606a-4e77-9b01-693f237b3e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:31.702318 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219aada5-606a-4e77-9b01-693f237b3e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:31.704359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219aada5-606a-4e77-9b01-693f237b3e8b admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:31.711959 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 952/1919] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:31.720443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe300e-8174-46e3-b3e9-99793e11059b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:31.722515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1dfe300e-8174-46e3-b3e9-99793e11059b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:31.737073 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 968/1920] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:31.741330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219aada5-606a-4e77-9b01-693f237b3e8b admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:31.754583 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219aada5-606a-4e77-9b01-693f237b3e8b admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:31.762440 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6089a537-833d-4bf5-bfad-4033d2ec51e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:31.764053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089a537-833d-4bf5-bfad-4033d2ec51e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:31.764344 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089a537-833d-4bf5-bfad-4033d2ec51e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:31.764620 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089a537-833d-4bf5-bfad-4033d2ec51e9 admin admin] PATH_INFO: `/v3/projects/f41b81967c824448ba06e5aecd60f3e0/users/1bfa1ee424574a60b11dff8934353329/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:31.766635 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6089a537-833d-4bf5-bfad-4033d2ec51e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f41b81967c824448ba06e5aecd60f3e0, user_id=1bfa1ee424574a60b11dff8934353329, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:31.771771 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6089a537-833d-4bf5-bfad-4033d2ec51e9 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:31.787896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 969/1921] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:31 2022] PUT /identity/v3/projects/f41b81967c824448ba06e5aecd60f3e0/users/1bfa1ee424574a60b11dff8934353329/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:35:31.796162 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe300e-8174-46e3-b3e9-99793e11059b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:31.802621 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d55d4-7045-4d93-bcc5-742ba037bc0b None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:31.802907 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d55d4-7045-4d93-bcc5-742ba037bc0b None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:31.803205 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d55d4-7045-4d93-bcc5-742ba037bc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:31.856500 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:31.859837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-629d55d4-7045-4d93-bcc5-742ba037bc0b None None] MFA Rules not processed for user `1bfa1ee424574a60b11dff8934353329`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:31.862566 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-629d55d4-7045-4d93-bcc5-742ba037bc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:31.870021 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fce038-2217-4dac-95cd-7175f26e79e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:31.871124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fce038-2217-4dac-95cd-7175f26e79e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:31.871538 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fce038-2217-4dac-95cd-7175f26e79e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:31.872804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fce038-2217-4dac-95cd-7175f26e79e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:31.951290 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 970/1922] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:31.951821 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fce038-2217-4dac-95cd-7175f26e79e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:31.953127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fce038-2217-4dac-95cd-7175f26e79e0 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:31.963226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d55d4-7045-4d93-bcc5-742ba037bc0b None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:31.964471 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 953/1923] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:35:31.964639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-629d55d4-7045-4d93-bcc5-742ba037bc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:32.023710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d55d4-7045-4d93-bcc5-742ba037bc0b None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:32.027486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482258a4-20bc-4a7c-a11d-d22e126ae0a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:32.028153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482258a4-20bc-4a7c-a11d-d22e126ae0a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:32.028437 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482258a4-20bc-4a7c-a11d-d22e126ae0a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:32.028717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482258a4-20bc-4a7c-a11d-d22e126ae0a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:32.039379 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482258a4-20bc-4a7c-a11d-d22e126ae0a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:32.040833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482258a4-20bc-4a7c-a11d-d22e126ae0a5 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:32.052307 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 971/1924] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:35:32.093539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fce038-2217-4dac-95cd-7175f26e79e0 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:32.094979 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-62fce038-2217-4dac-95cd-7175f26e79e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:32.107401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fce038-2217-4dac-95cd-7175f26e79e0 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:32.111002 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64a49e-0c8e-4dc1-a6dc-0cfc74df1228 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:32.112208 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64a49e-0c8e-4dc1-a6dc-0cfc74df1228 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:32.112497 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64a49e-0c8e-4dc1-a6dc-0cfc74df1228 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:32.112797 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64a49e-0c8e-4dc1-a6dc-0cfc74df1228 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:32.114622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3b64a49e-0c8e-4dc1-a6dc-0cfc74df1228 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:32.176177 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64a49e-0c8e-4dc1-a6dc-0cfc74df1228 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:32.177662 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64a49e-0c8e-4dc1-a6dc-0cfc74df1228 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:32.189182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 954/1925] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:32.851132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482258a4-20bc-4a7c-a11d-d22e126ae0a5 service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:32.853303 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-482258a4-20bc-4a7c-a11d-d22e126ae0a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:32.870823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482258a4-20bc-4a7c-a11d-d22e126ae0a5 service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:32.875647 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdba21-2980-4b57-9c23-2a32c0068ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:32.876460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdba21-2980-4b57-9c23-2a32c0068ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:32.877259 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdba21-2980-4b57-9c23-2a32c0068ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:32.877528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdba21-2980-4b57-9c23-2a32c0068ecb admin admin] PATH_INFO: `/v3/projects/fd51cdfd74ec448582ffad5306d9b705` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:32.878779 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdba21-2980-4b57-9c23-2a32c0068ecb admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd51cdfd74ec448582ffad5306d9b705)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:32.881221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdba21-2980-4b57-9c23-2a32c0068ecb admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:32.936217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:32.971730 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-97fdba21-2980-4b57-9c23-2a32c0068ecb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd51cdfd74ec448582ffad5306d9b705', 'request_id': 'req-97fdba21-2980-4b57-9c23-2a32c0068ecb'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:32.986014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-97fdba21-2980-4b57-9c23-2a32c0068ecb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd51cdfd74ec448582ffad5306d9b705', 'request_id': 'req-97fdba21-2980-4b57-9c23-2a32c0068ecb'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:32.993253 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-97fdba21-2980-4b57-9c23-2a32c0068ecb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd51cdfd74ec448582ffad5306d9b705', 'request_id': 'req-97fdba21-2980-4b57-9c23-2a32c0068ecb'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:33.004898 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-97fdba21-2980-4b57-9c23-2a32c0068ecb admin admin] The token cache is being invalidate because project fd51cdfd74ec448582ffad5306d9b705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:33.005992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-97fdba21-2980-4b57-9c23-2a32c0068ecb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:33.006451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-97fdba21-2980-4b57-9c23-2a32c0068ecb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:33.008463 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 972/1926] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:32 2022] DELETE /identity/v3/projects/fd51cdfd74ec448582ffad5306d9b705 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:33.012450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64a49e-0c8e-4dc1-a6dc-0cfc74df1228 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:33.025481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64a49e-0c8e-4dc1-a6dc-0cfc74df1228 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:33.097929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febffd38-3932-4b65-9f9b-af1366a323b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:33.098923 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febffd38-3932-4b65-9f9b-af1366a323b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:33.099350 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febffd38-3932-4b65-9f9b-af1366a323b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:33.099729 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febffd38-3932-4b65-9f9b-af1366a323b0 admin admin] PATH_INFO: `/v3/users/e112823d7b6f402d8cba2ddbd5799b71` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:33.101044 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febffd38-3932-4b65-9f9b-af1366a323b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e112823d7b6f402d8cba2ddbd5799b71)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:33.103142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febffd38-3932-4b65-9f9b-af1366a323b0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:33.145900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:33.157566 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-febffd38-3932-4b65-9f9b-af1366a323b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e112823d7b6f402d8cba2ddbd5799b71'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:33.165011 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-febffd38-3932-4b65-9f9b-af1366a323b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e112823d7b6f402d8cba2ddbd5799b71'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:33.165653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-febffd38-3932-4b65-9f9b-af1366a323b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e112823d7b6f402d8cba2ddbd5799b71'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:33.166286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-febffd38-3932-4b65-9f9b-af1366a323b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e112823d7b6f402d8cba2ddbd5799b71'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:33.179488 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-febffd38-3932-4b65-9f9b-af1366a323b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e112823d7b6f402d8cba2ddbd5799b71'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:33.192571 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 955/1927] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:33 2022] DELETE /identity/v3/users/e112823d7b6f402d8cba2ddbd5799b71 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:33.499991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdba21-2980-4b57-9c23-2a32c0068ecb admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:33.501586 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-97fdba21-2980-4b57-9c23-2a32c0068ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:33.518897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdba21-2980-4b57-9c23-2a32c0068ecb admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:33.588960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc39c6-3183-439b-aa29-213bf00ef89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:33.589997 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc39c6-3183-439b-aa29-213bf00ef89b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:33.590390 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc39c6-3183-439b-aa29-213bf00ef89b admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:33.590767 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc39c6-3183-439b-aa29-213bf00ef89b admin admin] PATH_INFO: `/v3/projects/9fe3d19ed3b84fb9b2db18c0652ccb3e` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:33.592224 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc39c6-3183-439b-aa29-213bf00ef89b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fe3d19ed3b84fb9b2db18c0652ccb3e)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:33.606593 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc39c6-3183-439b-aa29-213bf00ef89b admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:33.673549 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-73cc39c6-3183-439b-aa29-213bf00ef89b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fe3d19ed3b84fb9b2db18c0652ccb3e', 'request_id': 'req-73cc39c6-3183-439b-aa29-213bf00ef89b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:33.684545 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-73cc39c6-3183-439b-aa29-213bf00ef89b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fe3d19ed3b84fb9b2db18c0652ccb3e', 'request_id': 'req-73cc39c6-3183-439b-aa29-213bf00ef89b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:33.694783 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-73cc39c6-3183-439b-aa29-213bf00ef89b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fe3d19ed3b84fb9b2db18c0652ccb3e', 'request_id': 'req-73cc39c6-3183-439b-aa29-213bf00ef89b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:33.710789 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-73cc39c6-3183-439b-aa29-213bf00ef89b admin admin] The token cache is being invalidate because project 9fe3d19ed3b84fb9b2db18c0652ccb3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:33.711327 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-73cc39c6-3183-439b-aa29-213bf00ef89b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:33.712264 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-73cc39c6-3183-439b-aa29-213bf00ef89b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:33.714012 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 973/1928] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:33 2022] DELETE /identity/v3/projects/9fe3d19ed3b84fb9b2db18c0652ccb3e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:33.719281 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34e6de-7476-4fbb-97c8-b77321b27cf7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:33.719571 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34e6de-7476-4fbb-97c8-b77321b27cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:33.719846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34e6de-7476-4fbb-97c8-b77321b27cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:33.742453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-fb34e6de-7476-4fbb-97c8-b77321b27cf7 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:33.743969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fb34e6de-7476-4fbb-97c8-b77321b27cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:33.850671 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 956/1929] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:33.852478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02193443-0d7b-4298-b079-67f574ab06f0 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:33.852779 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02193443-0d7b-4298-b079-67f574ab06f0 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:33.853090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02193443-0d7b-4298-b079-67f574ab06f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:33.893155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-02193443-0d7b-4298-b079-67f574ab06f0 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:33.894910 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-02193443-0d7b-4298-b079-67f574ab06f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:33.911660 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 974/1930] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:33.915726 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34e6de-7476-4fbb-97c8-b77321b27cf7 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:33.928616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34e6de-7476-4fbb-97c8-b77321b27cf7 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:33.946522 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82221ee3-ab45-4b93-94bd-768dfa63ab7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:33.947266 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82221ee3-ab45-4b93-94bd-768dfa63ab7f admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:33.947552 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82221ee3-ab45-4b93-94bd-768dfa63ab7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:33.947827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82221ee3-ab45-4b93-94bd-768dfa63ab7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:33.948602 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82221ee3-ab45-4b93-94bd-768dfa63ab7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:35:33.948909 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82221ee3-ab45-4b93-94bd-768dfa63ab7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:33.954126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82221ee3-ab45-4b93-94bd-768dfa63ab7f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:33.983163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 957/1931] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:35:33.988481 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02193443-0d7b-4298-b079-67f574ab06f0 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:34.001280 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02193443-0d7b-4298-b079-67f574ab06f0 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:34.007263 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e79797-174b-4580-8896-ca223614d50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:34.008120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e79797-174b-4580-8896-ca223614d50f admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:34.008848 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e79797-174b-4580-8896-ca223614d50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:34.009550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e79797-174b-4580-8896-ca223614d50f admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:34.019690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e79797-174b-4580-8896-ca223614d50f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:34.022189 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e79797-174b-4580-8896-ca223614d50f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:34.065371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 975/1932] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:33 2022] POST /identity/v3/projects => generated 383 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:34.068955 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82221ee3-ab45-4b93-94bd-768dfa63ab7f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:34.082221 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82221ee3-ab45-4b93-94bd-768dfa63ab7f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:34.179474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c0987-4ac1-497e-a8f8-a11d918034a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:34.180259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c0987-4ac1-497e-a8f8-a11d918034a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:34.180782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c0987-4ac1-497e-a8f8-a11d918034a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:34.181161 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c0987-4ac1-497e-a8f8-a11d918034a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:34.181986 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c0987-4ac1-497e-a8f8-a11d918034a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:34.184159 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c0987-4ac1-497e-a8f8-a11d918034a2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:34.210628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 958/1933] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:34 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:34.215228 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e79797-174b-4580-8896-ca223614d50f admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:34.226693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e79797-174b-4580-8896-ca223614d50f admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:34.232144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779d515-4774-4f44-95e0-12e6c541a67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:34.232881 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779d515-4774-4f44-95e0-12e6c541a67c admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:34.233224 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779d515-4774-4f44-95e0-12e6c541a67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:34.233508 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779d515-4774-4f44-95e0-12e6c541a67c admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:34.234306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779d515-4774-4f44-95e0-12e6c541a67c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:34.236748 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779d515-4774-4f44-95e0-12e6c541a67c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:34.250044 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 976/1934] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:34.254872 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c0987-4ac1-497e-a8f8-a11d918034a2 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:34.270497 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c0987-4ac1-497e-a8f8-a11d918034a2 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:34.275837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba8c10-5641-4bdf-9ab1-ac04b92ddf10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:34.276526 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba8c10-5641-4bdf-9ab1-ac04b92ddf10 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:34.276807 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba8c10-5641-4bdf-9ab1-ac04b92ddf10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:34.277082 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba8c10-5641-4bdf-9ab1-ac04b92ddf10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:34.277823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba8c10-5641-4bdf-9ab1-ac04b92ddf10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:34.279556 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba8c10-5641-4bdf-9ab1-ac04b92ddf10 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:34.294251 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 959/1935] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:34.299210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779d515-4774-4f44-95e0-12e6c541a67c admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:34.312013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779d515-4774-4f44-95e0-12e6c541a67c admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:34.317018 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4a3b1-d1d4-4875-a938-0e53588ee291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:34.318007 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4a3b1-d1d4-4875-a938-0e53588ee291 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:34.318292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4a3b1-d1d4-4875-a938-0e53588ee291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:34.318570 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4a3b1-d1d4-4875-a938-0e53588ee291 admin admin] PATH_INFO: `/v3/projects/d075cba652e94aad9332d4d33e6f0298/users/d746d805537641b999e5bb1132e923dd/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:34.321189 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:34.335779 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4a3b1-d1d4-4875-a938-0e53588ee291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d075cba652e94aad9332d4d33e6f0298, user_id=d746d805537641b999e5bb1132e923dd, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:34.338460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4a3b1-d1d4-4875-a938-0e53588ee291 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:34.352223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 977/1936] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:34 2022] PUT /identity/v3/projects/d075cba652e94aad9332d4d33e6f0298/users/d746d805537641b999e5bb1132e923dd/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:34.363273 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a86ba0-28c5-4d31-a0fa-2c8401e7e2f5 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:34.363569 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a86ba0-28c5-4d31-a0fa-2c8401e7e2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:34.363852 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a86ba0-28c5-4d31-a0fa-2c8401e7e2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:34.394103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-45a86ba0-28c5-4d31-a0fa-2c8401e7e2f5 None None] MFA Rules not processed for user `d746d805537641b999e5bb1132e923dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:34.395353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-45a86ba0-28c5-4d31-a0fa-2c8401e7e2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:34.400607 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:34.468822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 960/1937] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:34.473939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4a3b1-d1d4-4875-a938-0e53588ee291 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:34.544637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4a3b1-d1d4-4875-a938-0e53588ee291 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:34.550312 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6597d-b54e-4bbd-aaaa-dc44efbdf3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:34.551042 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6597d-b54e-4bbd-aaaa-dc44efbdf3e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:34.551326 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6597d-b54e-4bbd-aaaa-dc44efbdf3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:34.551605 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6597d-b54e-4bbd-aaaa-dc44efbdf3e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:34.552331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6597d-b54e-4bbd-aaaa-dc44efbdf3e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:34.554258 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6597d-b54e-4bbd-aaaa-dc44efbdf3e9 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:34.582357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 978/1938] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:34 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:34.586777 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a86ba0-28c5-4d31-a0fa-2c8401e7e2f5 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:34.598565 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a86ba0-28c5-4d31-a0fa-2c8401e7e2f5 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:34.662569 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6df54-e2f7-4b9c-b5b0-d84b3533ccfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:34.663415 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6df54-e2f7-4b9c-b5b0-d84b3533ccfe admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:34.663766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6df54-e2f7-4b9c-b5b0-d84b3533ccfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:34.664125 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6df54-e2f7-4b9c-b5b0-d84b3533ccfe admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:34.664899 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6df54-e2f7-4b9c-b5b0-d84b3533ccfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:34.666927 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6df54-e2f7-4b9c-b5b0-d84b3533ccfe admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:34.692887 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 961/1939] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:34 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:34.693969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6597d-b54e-4bbd-aaaa-dc44efbdf3e9 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:34.704718 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6597d-b54e-4bbd-aaaa-dc44efbdf3e9 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:34.710263 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a46208-3dbf-43d2-a45c-5b6242595999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:34.710929 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a46208-3dbf-43d2-a45c-5b6242595999 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:34.711374 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a46208-3dbf-43d2-a45c-5b6242595999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:34.711654 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a46208-3dbf-43d2-a45c-5b6242595999 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:34.712377 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a46208-3dbf-43d2-a45c-5b6242595999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:34.713964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a46208-3dbf-43d2-a45c-5b6242595999 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:34.725356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 979/1940] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:34.729640 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6df54-e2f7-4b9c-b5b0-d84b3533ccfe admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:34.740051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6df54-e2f7-4b9c-b5b0-d84b3533ccfe admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:34.744932 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f52b2-e3b7-4267-8493-ba0866b2b701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:34.746318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f52b2-e3b7-4267-8493-ba0866b2b701 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:34.746614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f52b2-e3b7-4267-8493-ba0866b2b701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:34.746880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f52b2-e3b7-4267-8493-ba0866b2b701 admin admin] PATH_INFO: `/v3/projects/3439b18782df44feb977a9be5c0872f7/users/85bd1300e74942bea62e9034f4a6b621/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:34.761774 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f52b2-e3b7-4267-8493-ba0866b2b701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3439b18782df44feb977a9be5c0872f7, user_id=85bd1300e74942bea62e9034f4a6b621, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:34.764466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f52b2-e3b7-4267-8493-ba0866b2b701 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:34.778309 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 962/1941] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:34 2022] PUT /identity/v3/projects/3439b18782df44feb977a9be5c0872f7/users/85bd1300e74942bea62e9034f4a6b621/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:34.782358 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a46208-3dbf-43d2-a45c-5b6242595999 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:34.793475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a46208-3dbf-43d2-a45c-5b6242595999 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:34.856039 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1580be9-e2cd-4eff-a387-18d2c0ebcf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:34.856719 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1580be9-e2cd-4eff-a387-18d2c0ebcf63 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:34.856999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1580be9-e2cd-4eff-a387-18d2c0ebcf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:34.857271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1580be9-e2cd-4eff-a387-18d2c0ebcf63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:34.858005 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1580be9-e2cd-4eff-a387-18d2c0ebcf63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:34.859654 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1580be9-e2cd-4eff-a387-18d2c0ebcf63 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:34.869433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 980/1942] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:34.873628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f52b2-e3b7-4267-8493-ba0866b2b701 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:34.883917 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f52b2-e3b7-4267-8493-ba0866b2b701 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:34.888498 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93d100-9e97-463d-be32-5b1adde06859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:34.889378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93d100-9e97-463d-be32-5b1adde06859 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:34.889658 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93d100-9e97-463d-be32-5b1adde06859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:34.889932 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93d100-9e97-463d-be32-5b1adde06859 admin admin] PATH_INFO: `/v3/projects/3439b18782df44feb977a9be5c0872f7/users/85bd1300e74942bea62e9034f4a6b621/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:34.891960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba93d100-9e97-463d-be32-5b1adde06859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3439b18782df44feb977a9be5c0872f7, user_id=85bd1300e74942bea62e9034f4a6b621, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:34.895138 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba93d100-9e97-463d-be32-5b1adde06859 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:34.905758 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 963/1943] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:34 2022] PUT /identity/v3/projects/3439b18782df44feb977a9be5c0872f7/users/85bd1300e74942bea62e9034f4a6b621/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:34.917467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63726ec-e9f2-4efb-bb5a-2b3869f34afa None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:34.917754 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63726ec-e9f2-4efb-bb5a-2b3869f34afa None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:34.918034 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63726ec-e9f2-4efb-bb5a-2b3869f34afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:34.943740 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-d63726ec-e9f2-4efb-bb5a-2b3869f34afa None None] MFA Rules not processed for user `85bd1300e74942bea62e9034f4a6b621`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:34.944862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d63726ec-e9f2-4efb-bb5a-2b3869f34afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:34.992726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:35.027222 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 981/1944] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:35.044249 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93d100-9e97-463d-be32-5b1adde06859 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:35.046230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ba93d100-9e97-463d-be32-5b1adde06859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:35.106393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93d100-9e97-463d-be32-5b1adde06859 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:35.110609 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe6ce7-2193-47c4-827f-3e9b54538ce5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:35.111526 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe6ce7-2193-47c4-827f-3e9b54538ce5 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:35.111811 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe6ce7-2193-47c4-827f-3e9b54538ce5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:35.112084 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe6ce7-2193-47c4-827f-3e9b54538ce5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:35.122143 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfe6ce7-2193-47c4-827f-3e9b54538ce5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:35.123665 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfe6ce7-2193-47c4-827f-3e9b54538ce5 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:35.134736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 964/1945] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:35.212971 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63726ec-e9f2-4efb-bb5a-2b3869f34afa None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:35.214453 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d63726ec-e9f2-4efb-bb5a-2b3869f34afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:35.227360 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63726ec-e9f2-4efb-bb5a-2b3869f34afa None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:35.231070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a4a28-f81e-4b40-851a-2fcce1a2f8c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:35.231748 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a4a28-f81e-4b40-851a-2fcce1a2f8c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:35.232036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a4a28-f81e-4b40-851a-2fcce1a2f8c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:35.232350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a4a28-f81e-4b40-851a-2fcce1a2f8c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:35.304066 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a4a28-f81e-4b40-851a-2fcce1a2f8c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:35.305557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a4a28-f81e-4b40-851a-2fcce1a2f8c8 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:35.317204 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 982/1946] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:35:35.631598 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe6ce7-2193-47c4-827f-3e9b54538ce5 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:35.636102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:35.696220 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe6ce7-2193-47c4-827f-3e9b54538ce5 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:35.700666 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49f9b2-9264-4295-a3a7-d8ccd7b2532a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:35.701529 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49f9b2-9264-4295-a3a7-d8ccd7b2532a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:35.701812 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49f9b2-9264-4295-a3a7-d8ccd7b2532a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:35.702086 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49f9b2-9264-4295-a3a7-d8ccd7b2532a admin admin] PATH_INFO: `/v3/users/d746d805537641b999e5bb1132e923dd` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:35.703297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49f9b2-9264-4295-a3a7-d8ccd7b2532a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d746d805537641b999e5bb1132e923dd)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:35.705704 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49f9b2-9264-4295-a3a7-d8ccd7b2532a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:35.750383 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0c49f9b2-9264-4295-a3a7-d8ccd7b2532a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd746d805537641b999e5bb1132e923dd'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:35.757466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0c49f9b2-9264-4295-a3a7-d8ccd7b2532a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd746d805537641b999e5bb1132e923dd'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:35.758210 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0c49f9b2-9264-4295-a3a7-d8ccd7b2532a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd746d805537641b999e5bb1132e923dd'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:35.758818 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0c49f9b2-9264-4295-a3a7-d8ccd7b2532a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd746d805537641b999e5bb1132e923dd'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:35.772121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0c49f9b2-9264-4295-a3a7-d8ccd7b2532a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd746d805537641b999e5bb1132e923dd'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:35.784607 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 965/1947] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:35 2022] DELETE /identity/v3/users/d746d805537641b999e5bb1132e923dd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:35.789127 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d6f37-3e52-48e8-ac37-baaa5e9440b5 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:35.789419 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d6f37-3e52-48e8-ac37-baaa5e9440b5 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:35.789702 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d6f37-3e52-48e8-ac37-baaa5e9440b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:35.806479 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-186d6f37-3e52-48e8-ac37-baaa5e9440b5 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:35.807732 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-186d6f37-3e52-48e8-ac37-baaa5e9440b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:35.866406 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 983/1948] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:35.876054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49f9b2-9264-4295-a3a7-d8ccd7b2532a admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:35.878144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0c49f9b2-9264-4295-a3a7-d8ccd7b2532a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:35.934182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49f9b2-9264-4295-a3a7-d8ccd7b2532a admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:35.937761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912a5fb-3587-4073-93a0-801919d98135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:35.938409 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912a5fb-3587-4073-93a0-801919d98135 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:35.938679 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912a5fb-3587-4073-93a0-801919d98135 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:35.938993 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912a5fb-3587-4073-93a0-801919d98135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:35.948377 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912a5fb-3587-4073-93a0-801919d98135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:35.949736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912a5fb-3587-4073-93a0-801919d98135 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:35.959598 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 966/1949] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:36.230351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d6f37-3e52-48e8-ac37-baaa5e9440b5 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:36.232758 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-186d6f37-3e52-48e8-ac37-baaa5e9440b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:36.257081 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d6f37-3e52-48e8-ac37-baaa5e9440b5 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:36.265809 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee44581-d880-4c52-8844-7ea5ec7b427e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:36.267080 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee44581-d880-4c52-8844-7ea5ec7b427e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:36.267581 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee44581-d880-4c52-8844-7ea5ec7b427e admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:36.268632 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee44581-d880-4c52-8844-7ea5ec7b427e admin admin] PATH_INFO: `/v3/projects/d075cba652e94aad9332d4d33e6f0298` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:36.271714 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee44581-d880-4c52-8844-7ea5ec7b427e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d075cba652e94aad9332d4d33e6f0298)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:36.274706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee44581-d880-4c52-8844-7ea5ec7b427e admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:36.328652 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cee44581-d880-4c52-8844-7ea5ec7b427e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd075cba652e94aad9332d4d33e6f0298', 'request_id': 'req-cee44581-d880-4c52-8844-7ea5ec7b427e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:36.337680 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cee44581-d880-4c52-8844-7ea5ec7b427e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd075cba652e94aad9332d4d33e6f0298', 'request_id': 'req-cee44581-d880-4c52-8844-7ea5ec7b427e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:36.344537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cee44581-d880-4c52-8844-7ea5ec7b427e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd075cba652e94aad9332d4d33e6f0298', 'request_id': 'req-cee44581-d880-4c52-8844-7ea5ec7b427e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:36.353136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:36.357111 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cee44581-d880-4c52-8844-7ea5ec7b427e admin admin] The token cache is being invalidate because project d075cba652e94aad9332d4d33e6f0298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:36.357769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cee44581-d880-4c52-8844-7ea5ec7b427e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:36.358593 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cee44581-d880-4c52-8844-7ea5ec7b427e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:36.360747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 984/1950] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:36 2022] DELETE /identity/v3/projects/d075cba652e94aad9332d4d33e6f0298 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:36.365255 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912a5fb-3587-4073-93a0-801919d98135 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:36.375409 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912a5fb-3587-4073-93a0-801919d98135 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:36.402512 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:36.445288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d344258-7f99-4ec3-99e6-12a23ec6eeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:36.446299 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d344258-7f99-4ec3-99e6-12a23ec6eeb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:36.446702 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d344258-7f99-4ec3-99e6-12a23ec6eeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:36.447136 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d344258-7f99-4ec3-99e6-12a23ec6eeb6 admin admin] PATH_INFO: `/v3/users/85bd1300e74942bea62e9034f4a6b621` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:36.448533 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d344258-7f99-4ec3-99e6-12a23ec6eeb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85bd1300e74942bea62e9034f4a6b621)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:36.450369 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d344258-7f99-4ec3-99e6-12a23ec6eeb6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:36.507373 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3d344258-7f99-4ec3-99e6-12a23ec6eeb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85bd1300e74942bea62e9034f4a6b621'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:36.515337 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3d344258-7f99-4ec3-99e6-12a23ec6eeb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85bd1300e74942bea62e9034f4a6b621'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:36.515986 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3d344258-7f99-4ec3-99e6-12a23ec6eeb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85bd1300e74942bea62e9034f4a6b621'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:36.516596 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3d344258-7f99-4ec3-99e6-12a23ec6eeb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85bd1300e74942bea62e9034f4a6b621'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:36.545689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-3d344258-7f99-4ec3-99e6-12a23ec6eeb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85bd1300e74942bea62e9034f4a6b621'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:36.554201 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 967/1951] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:36 2022] DELETE /identity/v3/users/85bd1300e74942bea62e9034f4a6b621 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:36.928235 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee44581-d880-4c52-8844-7ea5ec7b427e admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:36.929738 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-cee44581-d880-4c52-8844-7ea5ec7b427e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:36.944592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee44581-d880-4c52-8844-7ea5ec7b427e admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:37.007262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa7537-4ada-4e25-8f49-2ff0918957d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:37.008333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa7537-4ada-4e25-8f49-2ff0918957d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:37.008619 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa7537-4ada-4e25-8f49-2ff0918957d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:37.008894 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa7537-4ada-4e25-8f49-2ff0918957d6 admin admin] PATH_INFO: `/v3/projects/3439b18782df44feb977a9be5c0872f7` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:37.010147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa7537-4ada-4e25-8f49-2ff0918957d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3439b18782df44feb977a9be5c0872f7)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:37.012242 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa7537-4ada-4e25-8f49-2ff0918957d6 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:37.063411 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-76fa7537-4ada-4e25-8f49-2ff0918957d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3439b18782df44feb977a9be5c0872f7', 'request_id': 'req-76fa7537-4ada-4e25-8f49-2ff0918957d6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:37.071709 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-76fa7537-4ada-4e25-8f49-2ff0918957d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3439b18782df44feb977a9be5c0872f7', 'request_id': 'req-76fa7537-4ada-4e25-8f49-2ff0918957d6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:37.078030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-76fa7537-4ada-4e25-8f49-2ff0918957d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3439b18782df44feb977a9be5c0872f7', 'request_id': 'req-76fa7537-4ada-4e25-8f49-2ff0918957d6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:37.089064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-76fa7537-4ada-4e25-8f49-2ff0918957d6 admin admin] The token cache is being invalidate because project 3439b18782df44feb977a9be5c0872f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:37.089348 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-76fa7537-4ada-4e25-8f49-2ff0918957d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:37.089991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-76fa7537-4ada-4e25-8f49-2ff0918957d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:37.091555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 985/1952] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:36 2022] DELETE /identity/v3/projects/3439b18782df44feb977a9be5c0872f7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:37.096997 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988dab9-414c-47b5-af32-103a36fd77ce None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:37.097373 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988dab9-414c-47b5-af32-103a36fd77ce None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:37.097735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988dab9-414c-47b5-af32-103a36fd77ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:37.120252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-9988dab9-414c-47b5-af32-103a36fd77ce None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:37.121578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9988dab9-414c-47b5-af32-103a36fd77ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:37.181279 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 968/1953] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:37.189305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5d841-1093-4511-bfdb-f50f1af49ff9 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:37.189604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5d841-1093-4511-bfdb-f50f1af49ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:37.189887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5d841-1093-4511-bfdb-f50f1af49ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:37.209417 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-09f5d841-1093-4511-bfdb-f50f1af49ff9 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:37.210653 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-09f5d841-1093-4511-bfdb-f50f1af49ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:37.218901 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 986/1954] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:37.224070 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988dab9-414c-47b5-af32-103a36fd77ce None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:37.233821 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988dab9-414c-47b5-af32-103a36fd77ce None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:37.238903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e167dcf-c354-4f36-bac8-c9cc16ff5107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:37.239746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e167dcf-c354-4f36-bac8-c9cc16ff5107 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:37.240083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e167dcf-c354-4f36-bac8-c9cc16ff5107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:37.240510 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e167dcf-c354-4f36-bac8-c9cc16ff5107 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:37.241261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e167dcf-c354-4f36-bac8-c9cc16ff5107 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:35:37.241570 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e167dcf-c354-4f36-bac8-c9cc16ff5107 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:37.243334 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e167dcf-c354-4f36-bac8-c9cc16ff5107 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:37.255725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 969/1955] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:35:37.260100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5d841-1093-4511-bfdb-f50f1af49ff9 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:37.270219 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5d841-1093-4511-bfdb-f50f1af49ff9 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:37.275274 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9a69a-185b-4848-87aa-9d9d72e3429c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:37.275933 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9a69a-185b-4848-87aa-9d9d72e3429c admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:37.276217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9a69a-185b-4848-87aa-9d9d72e3429c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:37.276494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9a69a-185b-4848-87aa-9d9d72e3429c admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:37.277208 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9a69a-185b-4848-87aa-9d9d72e3429c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:37.279159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9a69a-185b-4848-87aa-9d9d72e3429c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:37.306964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 987/1956] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:37 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:37.311802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e167dcf-c354-4f36-bac8-c9cc16ff5107 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:37.323193 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e167dcf-c354-4f36-bac8-c9cc16ff5107 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:37.358843 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:37.367055 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9a69a-185b-4848-87aa-9d9d72e3429c admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:37.369011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-fbf9a69a-185b-4848-87aa-9d9d72e3429c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:37.404778 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5d523-7984-4ee3-bca2-1ca2a307e7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:37.405884 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5d523-7984-4ee3-bca2-1ca2a307e7ca admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:37.406204 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5d523-7984-4ee3-bca2-1ca2a307e7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:37.406499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5d523-7984-4ee3-bca2-1ca2a307e7ca admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:37.407589 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5d523-7984-4ee3-bca2-1ca2a307e7ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:37.409845 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5d523-7984-4ee3-bca2-1ca2a307e7ca admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:37.438415 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 970/1957] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:37 2022] POST /identity/v3/users => generated 289 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:37.444263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5d523-7984-4ee3-bca2-1ca2a307e7ca admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:37.470084 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9a69a-185b-4848-87aa-9d9d72e3429c admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:37.477321 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5d523-7984-4ee3-bca2-1ca2a307e7ca admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:37.479306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dffe3a-c561-4026-870f-b7da9103747c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:37.480424 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dffe3a-c561-4026-870f-b7da9103747c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:37.480889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dffe3a-c561-4026-870f-b7da9103747c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:37.481190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dffe3a-c561-4026-870f-b7da9103747c admin admin] PATH_INFO: `/v3/users/09e00efecab742d78df7f9aa6ae81a85` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:37.482788 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91b743-acda-4572-895b-e33c0723b3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:37.483557 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91b743-acda-4572-895b-e33c0723b3c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:37.483827 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dffe3a-c561-4026-870f-b7da9103747c admin admin] RBAC: Authorizing `identity:delete_user(user_id=09e00efecab742d78df7f9aa6ae81a85)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:37.483962 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91b743-acda-4572-895b-e33c0723b3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:37.484355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91b743-acda-4572-895b-e33c0723b3c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:37.485166 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e91b743-acda-4572-895b-e33c0723b3c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:37.486145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dffe3a-c561-4026-870f-b7da9103747c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:37.487064 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e91b743-acda-4572-895b-e33c0723b3c3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:37.489947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:37.507079 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 971/1958] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:37.508172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91b743-acda-4572-895b-e33c0723b3c3 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:37.522610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91b743-acda-4572-895b-e33c0723b3c3 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:37.529101 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d318ca4-084c-408d-8fb2-11ee1cf12d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:37.530055 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d318ca4-084c-408d-8fb2-11ee1cf12d0d admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:37.530340 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d318ca4-084c-408d-8fb2-11ee1cf12d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:37.530620 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d318ca4-084c-408d-8fb2-11ee1cf12d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:37.531405 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d318ca4-084c-408d-8fb2-11ee1cf12d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:37.533239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d318ca4-084c-408d-8fb2-11ee1cf12d0d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:37.552789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 972/1959] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:37.556716 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d318ca4-084c-408d-8fb2-11ee1cf12d0d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:37.573273 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d318ca4-084c-408d-8fb2-11ee1cf12d0d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:37.580491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f5221-ec41-48c0-b6c0-4921a0b42b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:37.581652 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f5221-ec41-48c0-b6c0-4921a0b42b96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:37.581947 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f5221-ec41-48c0-b6c0-4921a0b42b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:37.582233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f5221-ec41-48c0-b6c0-4921a0b42b96 admin admin] PATH_INFO: `/v3/projects/218c95949b8843cbbac812c44625e517/users/b5e6ad9771e64da29fed2ed07c64d10e/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:37.594052 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-19dffe3a-c561-4026-870f-b7da9103747c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09e00efecab742d78df7f9aa6ae81a85'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:37.607490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-19dffe3a-c561-4026-870f-b7da9103747c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09e00efecab742d78df7f9aa6ae81a85'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:37.618950 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-19dffe3a-c561-4026-870f-b7da9103747c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09e00efecab742d78df7f9aa6ae81a85'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:37.620234 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-19dffe3a-c561-4026-870f-b7da9103747c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09e00efecab742d78df7f9aa6ae81a85'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:37.621899 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-19dffe3a-c561-4026-870f-b7da9103747c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09e00efecab742d78df7f9aa6ae81a85'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:37.644805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f5221-ec41-48c0-b6c0-4921a0b42b96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=218c95949b8843cbbac812c44625e517, user_id=b5e6ad9771e64da29fed2ed07c64d10e, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:37.655944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f5221-ec41-48c0-b6c0-4921a0b42b96 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:37.693977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 973/1960] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:37 2022] PUT /identity/v3/projects/218c95949b8843cbbac812c44625e517/users/b5e6ad9771e64da29fed2ed07c64d10e/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:37.699570 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 988/1961] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:37 2022] DELETE /identity/v3/users/09e00efecab742d78df7f9aa6ae81a85 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:35:37.707120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb22c29-86c4-4129-965d-7acb06a350e2 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:37.707420 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb22c29-86c4-4129-965d-7acb06a350e2 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:37.707751 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb22c29-86c4-4129-965d-7acb06a350e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:37.708718 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4afb5-0762-42dc-b185-acabf642183d None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:37.708897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4afb5-0762-42dc-b185-acabf642183d None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:37.709187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4afb5-0762-42dc-b185-acabf642183d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:37.731629 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8cb22c29-86c4-4129-965d-7acb06a350e2 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:37.733741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8cb22c29-86c4-4129-965d-7acb06a350e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:37.755068 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-e8e4afb5-0762-42dc-b185-acabf642183d None None] MFA Rules not processed for user `b5e6ad9771e64da29fed2ed07c64d10e`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:37.756947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e8e4afb5-0762-42dc-b185-acabf642183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:37.857478 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 974/1962] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:37.864447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb22c29-86c4-4129-965d-7acb06a350e2 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:37.866069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8cb22c29-86c4-4129-965d-7acb06a350e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:37.887938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 989/1963] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:35:37.900689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4afb5-0762-42dc-b185-acabf642183d None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:37.902079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e8e4afb5-0762-42dc-b185-acabf642183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:37.970913 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb22c29-86c4-4129-965d-7acb06a350e2 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:37.975512 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4afb5-0762-42dc-b185-acabf642183d None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:37.979828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2922d-6b74-435f-b378-7bff13ee5436 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:37.980532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2922d-6b74-435f-b378-7bff13ee5436 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:37.980679 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd9103-19bc-469d-b791-fde0c60ef2d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:37.980817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2922d-6b74-435f-b378-7bff13ee5436 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:37.981091 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2922d-6b74-435f-b378-7bff13ee5436 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:37.981829 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd9103-19bc-469d-b791-fde0c60ef2d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:37.982333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd9103-19bc-469d-b791-fde0c60ef2d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:37.982738 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd9103-19bc-469d-b791-fde0c60ef2d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:37.997546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd9103-19bc-469d-b791-fde0c60ef2d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:37.999201 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd9103-19bc-469d-b791-fde0c60ef2d4 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:38.004122 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2922d-6b74-435f-b378-7bff13ee5436 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:38.006970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2922d-6b74-435f-b378-7bff13ee5436 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:38.010892 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 975/1964] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:35:38.024653 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 990/1965] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:35:38.374473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd9103-19bc-469d-b791-fde0c60ef2d4 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:38.375970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5bfd9103-19bc-469d-b791-fde0c60ef2d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:38.396585 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd9103-19bc-469d-b791-fde0c60ef2d4 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:38.403641 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1419b-19e0-491a-a16a-78506f6e7d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:38.404717 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1419b-19e0-491a-a16a-78506f6e7d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:38.405184 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1419b-19e0-491a-a16a-78506f6e7d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:38.405708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1419b-19e0-491a-a16a-78506f6e7d13 admin admin] PATH_INFO: `/v3/projects/55ec62064c8d4380996ab0a5de9c0b38` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:38.406965 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f1419b-19e0-491a-a16a-78506f6e7d13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55ec62064c8d4380996ab0a5de9c0b38)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:38.409163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f1419b-19e0-491a-a16a-78506f6e7d13 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:38.424357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:38.469243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-13f1419b-19e0-491a-a16a-78506f6e7d13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55ec62064c8d4380996ab0a5de9c0b38', 'request_id': 'req-13f1419b-19e0-491a-a16a-78506f6e7d13'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:38.475702 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-13f1419b-19e0-491a-a16a-78506f6e7d13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55ec62064c8d4380996ab0a5de9c0b38', 'request_id': 'req-13f1419b-19e0-491a-a16a-78506f6e7d13'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:38.481169 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-13f1419b-19e0-491a-a16a-78506f6e7d13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55ec62064c8d4380996ab0a5de9c0b38', 'request_id': 'req-13f1419b-19e0-491a-a16a-78506f6e7d13'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:38.493848 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-13f1419b-19e0-491a-a16a-78506f6e7d13 admin admin] The token cache is being invalidate because project 55ec62064c8d4380996ab0a5de9c0b38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:38.494133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-13f1419b-19e0-491a-a16a-78506f6e7d13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:38.494750 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-13f1419b-19e0-491a-a16a-78506f6e7d13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:38.496331 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 976/1966] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:38 2022] DELETE /identity/v3/projects/55ec62064c8d4380996ab0a5de9c0b38 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:38.500268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2922d-6b74-435f-b378-7bff13ee5436 service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:38.511806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2922d-6b74-435f-b378-7bff13ee5436 service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:38.520529 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:38.570985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862e24c-67e8-4361-a309-182cd60f3a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:38.571871 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862e24c-67e8-4361-a309-182cd60f3a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:38.572158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862e24c-67e8-4361-a309-182cd60f3a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:38.572436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862e24c-67e8-4361-a309-182cd60f3a8b admin admin] PATH_INFO: `/v3/users/dbb61a7f38394fc298cabf78b499106e` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:38.573626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862e24c-67e8-4361-a309-182cd60f3a8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbb61a7f38394fc298cabf78b499106e)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:38.575591 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862e24c-67e8-4361-a309-182cd60f3a8b admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:38.621106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7862e24c-67e8-4361-a309-182cd60f3a8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbb61a7f38394fc298cabf78b499106e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:38.631709 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7862e24c-67e8-4361-a309-182cd60f3a8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbb61a7f38394fc298cabf78b499106e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:38.639746 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7862e24c-67e8-4361-a309-182cd60f3a8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbb61a7f38394fc298cabf78b499106e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:38.640414 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7862e24c-67e8-4361-a309-182cd60f3a8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbb61a7f38394fc298cabf78b499106e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:38.641165 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7862e24c-67e8-4361-a309-182cd60f3a8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbb61a7f38394fc298cabf78b499106e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:38.658616 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 991/1967] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:38 2022] DELETE /identity/v3/users/dbb61a7f38394fc298cabf78b499106e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:38.961234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1419b-19e0-491a-a16a-78506f6e7d13 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:38.963147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-13f1419b-19e0-491a-a16a-78506f6e7d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:38.978498 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1419b-19e0-491a-a16a-78506f6e7d13 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:39.085414 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b006c0a-f602-4b9b-94d1-d8b83d5a1fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:39.086448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b006c0a-f602-4b9b-94d1-d8b83d5a1fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:39.086917 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b006c0a-f602-4b9b-94d1-d8b83d5a1fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:39.087663 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b006c0a-f602-4b9b-94d1-d8b83d5a1fc3 admin admin] PATH_INFO: `/v3/projects/6a48e3d9d3c54088b99685ce39d9910e` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:39.090225 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b006c0a-f602-4b9b-94d1-d8b83d5a1fc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a48e3d9d3c54088b99685ce39d9910e)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:39.092889 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b006c0a-f602-4b9b-94d1-d8b83d5a1fc3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:39.200228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9b006c0a-f602-4b9b-94d1-d8b83d5a1fc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a48e3d9d3c54088b99685ce39d9910e', 'request_id': 'req-9b006c0a-f602-4b9b-94d1-d8b83d5a1fc3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:39.223114 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9b006c0a-f602-4b9b-94d1-d8b83d5a1fc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a48e3d9d3c54088b99685ce39d9910e', 'request_id': 'req-9b006c0a-f602-4b9b-94d1-d8b83d5a1fc3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:39.237815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9b006c0a-f602-4b9b-94d1-d8b83d5a1fc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a48e3d9d3c54088b99685ce39d9910e', 'request_id': 'req-9b006c0a-f602-4b9b-94d1-d8b83d5a1fc3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:39.272884 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9b006c0a-f602-4b9b-94d1-d8b83d5a1fc3 admin admin] The token cache is being invalidate because project 6a48e3d9d3c54088b99685ce39d9910e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:39.275951 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9b006c0a-f602-4b9b-94d1-d8b83d5a1fc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:39.281218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9b006c0a-f602-4b9b-94d1-d8b83d5a1fc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:39.283989 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 977/1968] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:38 2022] DELETE /identity/v3/projects/6a48e3d9d3c54088b99685ce39d9910e => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:39.288025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862e24c-67e8-4361-a309-182cd60f3a8b admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:39.291490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-7862e24c-67e8-4361-a309-182cd60f3a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:39.306763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862e24c-67e8-4361-a309-182cd60f3a8b admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:39.455192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82996780-1aa2-4c43-8479-9c8eadb8fd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:39.457920 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82996780-1aa2-4c43-8479-9c8eadb8fd2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:39.458206 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82996780-1aa2-4c43-8479-9c8eadb8fd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:39.458483 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82996780-1aa2-4c43-8479-9c8eadb8fd2c admin admin] PATH_INFO: `/v3/users/1bfa1ee424574a60b11dff8934353329` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:39.471663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82996780-1aa2-4c43-8479-9c8eadb8fd2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bfa1ee424574a60b11dff8934353329)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:39.471663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82996780-1aa2-4c43-8479-9c8eadb8fd2c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:39.573697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:39.600561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-82996780-1aa2-4c43-8479-9c8eadb8fd2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bfa1ee424574a60b11dff8934353329'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:39.614215 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-82996780-1aa2-4c43-8479-9c8eadb8fd2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bfa1ee424574a60b11dff8934353329'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:39.631223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-82996780-1aa2-4c43-8479-9c8eadb8fd2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bfa1ee424574a60b11dff8934353329'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:39.632947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-82996780-1aa2-4c43-8479-9c8eadb8fd2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bfa1ee424574a60b11dff8934353329'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:39.634135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-82996780-1aa2-4c43-8479-9c8eadb8fd2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bfa1ee424574a60b11dff8934353329'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:39.677134 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 992/1969] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:39 2022] DELETE /identity/v3/users/1bfa1ee424574a60b11dff8934353329 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:40.684744 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b006c0a-f602-4b9b-94d1-d8b83d5a1fc3 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:40.686388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9b006c0a-f602-4b9b-94d1-d8b83d5a1fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:40.702753 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b006c0a-f602-4b9b-94d1-d8b83d5a1fc3 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:40.752666 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:40.790861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe60bf-d07f-45dd-a1fb-32ff60d70ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:40.792185 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe60bf-d07f-45dd-a1fb-32ff60d70ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:40.792472 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe60bf-d07f-45dd-a1fb-32ff60d70ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:40.792748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe60bf-d07f-45dd-a1fb-32ff60d70ab6 admin admin] PATH_INFO: `/v3/projects/f41b81967c824448ba06e5aecd60f3e0` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:40.793983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe60bf-d07f-45dd-a1fb-32ff60d70ab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f41b81967c824448ba06e5aecd60f3e0)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:40.795960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe60bf-d07f-45dd-a1fb-32ff60d70ab6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:40.859777 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-25fe60bf-d07f-45dd-a1fb-32ff60d70ab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f41b81967c824448ba06e5aecd60f3e0', 'request_id': 'req-25fe60bf-d07f-45dd-a1fb-32ff60d70ab6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:40.867066 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-25fe60bf-d07f-45dd-a1fb-32ff60d70ab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f41b81967c824448ba06e5aecd60f3e0', 'request_id': 'req-25fe60bf-d07f-45dd-a1fb-32ff60d70ab6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:40.873334 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-25fe60bf-d07f-45dd-a1fb-32ff60d70ab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f41b81967c824448ba06e5aecd60f3e0', 'request_id': 'req-25fe60bf-d07f-45dd-a1fb-32ff60d70ab6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:40.895950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-25fe60bf-d07f-45dd-a1fb-32ff60d70ab6 admin admin] The token cache is being invalidate because project f41b81967c824448ba06e5aecd60f3e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:40.896670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-25fe60bf-d07f-45dd-a1fb-32ff60d70ab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:40.897397 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-25fe60bf-d07f-45dd-a1fb-32ff60d70ab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:40.899274 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 978/1970] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:40 2022] DELETE /identity/v3/projects/f41b81967c824448ba06e5aecd60f3e0 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:40.905175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4debc9-20db-40c5-8e45-489dec145c27 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:40.905701 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4debc9-20db-40c5-8e45-489dec145c27 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:40.905781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4debc9-20db-40c5-8e45-489dec145c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:40.929457 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-ce4debc9-20db-40c5-8e45-489dec145c27 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:40.931210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ce4debc9-20db-40c5-8e45-489dec145c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:41.000315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 993/1971] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:41.009313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825dd18-d735-4721-a1b3-3c0e5490bd45 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:41.009607 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825dd18-d735-4721-a1b3-3c0e5490bd45 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:41.009882 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825dd18-d735-4721-a1b3-3c0e5490bd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:41.030291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c825dd18-d735-4721-a1b3-3c0e5490bd45 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:41.031778 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c825dd18-d735-4721-a1b3-3c0e5490bd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:41.040607 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 979/1972] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:41.044873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4debc9-20db-40c5-8e45-489dec145c27 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:41.057525 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4debc9-20db-40c5-8e45-489dec145c27 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:41.062921 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fec293-0147-4f75-a075-6a2b5470b5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:41.063620 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fec293-0147-4f75-a075-6a2b5470b5c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:41.063928 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fec293-0147-4f75-a075-6a2b5470b5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:41.064211 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fec293-0147-4f75-a075-6a2b5470b5c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:41.064956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fec293-0147-4f75-a075-6a2b5470b5c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:35:41.065265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fec293-0147-4f75-a075-6a2b5470b5c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:41.066868 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fec293-0147-4f75-a075-6a2b5470b5c3 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:41.079375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 994/1973] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:35:41.084113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825dd18-d735-4721-a1b3-3c0e5490bd45 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:41.093497 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825dd18-d735-4721-a1b3-3c0e5490bd45 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:41.098630 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a31e3-a457-4e01-8f9d-0e48c1da6d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:41.099318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a31e3-a457-4e01-8f9d-0e48c1da6d96 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:41.099600 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a31e3-a457-4e01-8f9d-0e48c1da6d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:41.099872 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a31e3-a457-4e01-8f9d-0e48c1da6d96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:41.100571 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a31e3-a457-4e01-8f9d-0e48c1da6d96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:41.102676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a31e3-a457-4e01-8f9d-0e48c1da6d96 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:41.131009 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 980/1974] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:41 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:41.135495 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fec293-0147-4f75-a075-6a2b5470b5c3 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:41.146687 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fec293-0147-4f75-a075-6a2b5470b5c3 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:41.204960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfec83-c149-40a8-a7b1-35e69e2a997b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:41.205643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfec83-c149-40a8-a7b1-35e69e2a997b admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:41.205929 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfec83-c149-40a8-a7b1-35e69e2a997b admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:41.206211 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfec83-c149-40a8-a7b1-35e69e2a997b admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:41.207230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cfec83-c149-40a8-a7b1-35e69e2a997b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:41.209164 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cfec83-c149-40a8-a7b1-35e69e2a997b admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:41.232982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 995/1975] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:41 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:41.234309 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a31e3-a457-4e01-8f9d-0e48c1da6d96 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:41.245583 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a31e3-a457-4e01-8f9d-0e48c1da6d96 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:41.250694 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6fc8bc-2866-457f-b612-49c73c410373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:41.251425 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fc8bc-2866-457f-b612-49c73c410373 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:41.251691 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fc8bc-2866-457f-b612-49c73c410373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:41.251963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fc8bc-2866-457f-b612-49c73c410373 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:41.252690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6fc8bc-2866-457f-b612-49c73c410373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:41.254315 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6fc8bc-2866-457f-b612-49c73c410373 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:41.268697 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 981/1976] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:41.270158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfec83-c149-40a8-a7b1-35e69e2a997b admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:41.280822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfec83-c149-40a8-a7b1-35e69e2a997b admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:41.286063 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af8345-a7cd-4115-8a0d-028a1def60b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:41.286732 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af8345-a7cd-4115-8a0d-028a1def60b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:41.287051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af8345-a7cd-4115-8a0d-028a1def60b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:41.287484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af8345-a7cd-4115-8a0d-028a1def60b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:41.288229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af8345-a7cd-4115-8a0d-028a1def60b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:41.289833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af8345-a7cd-4115-8a0d-028a1def60b0 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:41.300814 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 996/1977] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:41.305001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6fc8bc-2866-457f-b612-49c73c410373 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:41.315606 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6fc8bc-2866-457f-b612-49c73c410373 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:41.320326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c37abb-92c9-4c70-aabd-24823d25c5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:41.321229 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c37abb-92c9-4c70-aabd-24823d25c5ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:41.321515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c37abb-92c9-4c70-aabd-24823d25c5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:41.321797 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c37abb-92c9-4c70-aabd-24823d25c5ee admin admin] PATH_INFO: `/v3/projects/4f603ad18d384ed191333456eaec85e8/users/f13fa04598504ae78d7ce1f1a42baf94/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:41.339341 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c37abb-92c9-4c70-aabd-24823d25c5ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f603ad18d384ed191333456eaec85e8, user_id=f13fa04598504ae78d7ce1f1a42baf94, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:41.342018 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c37abb-92c9-4c70-aabd-24823d25c5ee admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:41.361986 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 982/1978] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:41 2022] PUT /identity/v3/projects/4f603ad18d384ed191333456eaec85e8/users/f13fa04598504ae78d7ce1f1a42baf94/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:41.368376 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f4283-1d50-40a9-872b-0331916ffba5 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:41.368671 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f4283-1d50-40a9-872b-0331916ffba5 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:41.369063 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f4283-1d50-40a9-872b-0331916ffba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:41.398081 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-742f4283-1d50-40a9-872b-0331916ffba5 None None] MFA Rules not processed for user `f13fa04598504ae78d7ce1f1a42baf94`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:41.399351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-742f4283-1d50-40a9-872b-0331916ffba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:41.406589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:41.472755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 997/1979] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:41.484655 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c37abb-92c9-4c70-aabd-24823d25c5ee admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:41.487075 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-94c37abb-92c9-4c70-aabd-24823d25c5ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:41.546005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c37abb-92c9-4c70-aabd-24823d25c5ee admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:41.550512 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24918db-9b6f-4471-87dc-44a1f33c6ef0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:41.551181 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24918db-9b6f-4471-87dc-44a1f33c6ef0 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:41.551461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24918db-9b6f-4471-87dc-44a1f33c6ef0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:41.551735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24918db-9b6f-4471-87dc-44a1f33c6ef0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:41.560958 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24918db-9b6f-4471-87dc-44a1f33c6ef0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:41.562412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24918db-9b6f-4471-87dc-44a1f33c6ef0 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:41.568698 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:41.574538 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 983/1980] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:41.594399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f4283-1d50-40a9-872b-0331916ffba5 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:41.659780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f4283-1d50-40a9-872b-0331916ffba5 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:41.664415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e634c0-a087-4b14-8ffb-932c4a3027f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:41.665294 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e634c0-a087-4b14-8ffb-932c4a3027f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:41.665575 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e634c0-a087-4b14-8ffb-932c4a3027f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:41.665846 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e634c0-a087-4b14-8ffb-932c4a3027f5 admin admin] PATH_INFO: `/v3/users/f13fa04598504ae78d7ce1f1a42baf94` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:41.667681 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e634c0-a087-4b14-8ffb-932c4a3027f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f13fa04598504ae78d7ce1f1a42baf94)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:41.669598 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e634c0-a087-4b14-8ffb-932c4a3027f5 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:41.720826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e1e634c0-a087-4b14-8ffb-932c4a3027f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f13fa04598504ae78d7ce1f1a42baf94'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:41.729784 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e1e634c0-a087-4b14-8ffb-932c4a3027f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f13fa04598504ae78d7ce1f1a42baf94'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:41.740027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e1e634c0-a087-4b14-8ffb-932c4a3027f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f13fa04598504ae78d7ce1f1a42baf94'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:41.741260 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e1e634c0-a087-4b14-8ffb-932c4a3027f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f13fa04598504ae78d7ce1f1a42baf94'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:41.741892 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e1e634c0-a087-4b14-8ffb-932c4a3027f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f13fa04598504ae78d7ce1f1a42baf94'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:41.766672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 998/1981] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:41 2022] DELETE /identity/v3/users/f13fa04598504ae78d7ce1f1a42baf94 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:41.771985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f1918-aa11-4509-95ea-007aba355ee7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:41.772545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f1918-aa11-4509-95ea-007aba355ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:41.772545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f1918-aa11-4509-95ea-007aba355ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:41.791680 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-0c6f1918-aa11-4509-95ea-007aba355ee7 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:41.801561 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0c6f1918-aa11-4509-95ea-007aba355ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:41.866212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 984/1982] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:41.876522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e634c0-a087-4b14-8ffb-932c4a3027f5 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:41.878527 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e1e634c0-a087-4b14-8ffb-932c4a3027f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:41.924063 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:41.956318 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e634c0-a087-4b14-8ffb-932c4a3027f5 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:41.960295 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972506c-9920-4f7d-8b0e-0ce580202f89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:41.960897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972506c-9920-4f7d-8b0e-0ce580202f89 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:41.961325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972506c-9920-4f7d-8b0e-0ce580202f89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:41.961494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972506c-9920-4f7d-8b0e-0ce580202f89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:41.981701 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972506c-9920-4f7d-8b0e-0ce580202f89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:41.983195 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972506c-9920-4f7d-8b0e-0ce580202f89 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:41.994772 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 999/1983] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:42.332940 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f1918-aa11-4509-95ea-007aba355ee7 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:42.334547 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0c6f1918-aa11-4509-95ea-007aba355ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:42.349248 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f1918-aa11-4509-95ea-007aba355ee7 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:42.354764 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b090a-3d3c-4a5b-a60f-1f687e27cafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:42.357087 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b090a-3d3c-4a5b-a60f-1f687e27cafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:42.357407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b090a-3d3c-4a5b-a60f-1f687e27cafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:42.357687 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b090a-3d3c-4a5b-a60f-1f687e27cafc admin admin] PATH_INFO: `/v3/projects/4f603ad18d384ed191333456eaec85e8` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:42.359216 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b090a-3d3c-4a5b-a60f-1f687e27cafc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f603ad18d384ed191333456eaec85e8)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:42.361478 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b090a-3d3c-4a5b-a60f-1f687e27cafc admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:42.411407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e32b090a-3d3c-4a5b-a60f-1f687e27cafc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f603ad18d384ed191333456eaec85e8', 'request_id': 'req-e32b090a-3d3c-4a5b-a60f-1f687e27cafc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:42.417627 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e32b090a-3d3c-4a5b-a60f-1f687e27cafc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f603ad18d384ed191333456eaec85e8', 'request_id': 'req-e32b090a-3d3c-4a5b-a60f-1f687e27cafc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:42.423667 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e32b090a-3d3c-4a5b-a60f-1f687e27cafc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f603ad18d384ed191333456eaec85e8', 'request_id': 'req-e32b090a-3d3c-4a5b-a60f-1f687e27cafc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:42.436701 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e32b090a-3d3c-4a5b-a60f-1f687e27cafc admin admin] The token cache is being invalidate because project 4f603ad18d384ed191333456eaec85e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:42.436988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e32b090a-3d3c-4a5b-a60f-1f687e27cafc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:42.437594 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e32b090a-3d3c-4a5b-a60f-1f687e27cafc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:42.439179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 985/1984] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:42 2022] DELETE /identity/v3/projects/4f603ad18d384ed191333456eaec85e8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:42.444238 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f654bf-b2fd-411b-9c3b-34ab646b4347 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:42.444452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f654bf-b2fd-411b-9c3b-34ab646b4347 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:42.444730 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f654bf-b2fd-411b-9c3b-34ab646b4347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:42.462486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-b2f654bf-b2fd-411b-9c3b-34ab646b4347 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:42.463706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b2f654bf-b2fd-411b-9c3b-34ab646b4347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:42.522747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1000/1985] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:42.530590 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad11c1e-4837-43b2-8496-3e0066da2132 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:42.530914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad11c1e-4837-43b2-8496-3e0066da2132 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:42.531225 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad11c1e-4837-43b2-8496-3e0066da2132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:42.549731 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-0ad11c1e-4837-43b2-8496-3e0066da2132 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:42.551003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0ad11c1e-4837-43b2-8496-3e0066da2132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:42.558853 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 986/1986] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:42.563130 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f654bf-b2fd-411b-9c3b-34ab646b4347 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:42.573680 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f654bf-b2fd-411b-9c3b-34ab646b4347 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:42.579389 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d8fda-c042-41bd-b3cf-7bc9194f2620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:42.581194 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d8fda-c042-41bd-b3cf-7bc9194f2620 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:42.581710 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d8fda-c042-41bd-b3cf-7bc9194f2620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:42.582230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d8fda-c042-41bd-b3cf-7bc9194f2620 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:42.583498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d8fda-c042-41bd-b3cf-7bc9194f2620 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:35:42.583821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d8fda-c042-41bd-b3cf-7bc9194f2620 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:42.585454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d8fda-c042-41bd-b3cf-7bc9194f2620 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:42.597578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1001/1987] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:35:42.602142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad11c1e-4837-43b2-8496-3e0066da2132 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:42.612305 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad11c1e-4837-43b2-8496-3e0066da2132 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:42.617017 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c909a-48bd-47ab-9596-71d6fb047176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:42.617673 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c909a-48bd-47ab-9596-71d6fb047176 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:42.617957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c909a-48bd-47ab-9596-71d6fb047176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:42.618235 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c909a-48bd-47ab-9596-71d6fb047176 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:42.618966 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c909a-48bd-47ab-9596-71d6fb047176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:42.620868 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c909a-48bd-47ab-9596-71d6fb047176 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:42.648501 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 987/1988] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:42 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:42.652647 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d8fda-c042-41bd-b3cf-7bc9194f2620 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:42.662606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d8fda-c042-41bd-b3cf-7bc9194f2620 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:42.720533 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944e0cd-d727-4f74-ba07-b9364ba960c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:42.721123 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944e0cd-d727-4f74-ba07-b9364ba960c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:42.721411 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944e0cd-d727-4f74-ba07-b9364ba960c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:42.721690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944e0cd-d727-4f74-ba07-b9364ba960c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:42.722395 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944e0cd-d727-4f74-ba07-b9364ba960c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:42.724349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944e0cd-d727-4f74-ba07-b9364ba960c0 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:42.745594 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1002/1989] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:42 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:42.746830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c909a-48bd-47ab-9596-71d6fb047176 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:42.758340 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c909a-48bd-47ab-9596-71d6fb047176 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:42.763116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da2ead-2bfd-4109-b565-fbec325ec1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:42.763775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da2ead-2bfd-4109-b565-fbec325ec1e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:42.764064 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da2ead-2bfd-4109-b565-fbec325ec1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:42.764345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da2ead-2bfd-4109-b565-fbec325ec1e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:42.765077 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da2ead-2bfd-4109-b565-fbec325ec1e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:42.766683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da2ead-2bfd-4109-b565-fbec325ec1e6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:42.778484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 988/1990] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:42.782142 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944e0cd-d727-4f74-ba07-b9364ba960c0 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:42.792130 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944e0cd-d727-4f74-ba07-b9364ba960c0 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:42.797032 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97338a83-5246-4538-a8a8-b45f05701955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:42.797693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97338a83-5246-4538-a8a8-b45f05701955 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:42.797978 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97338a83-5246-4538-a8a8-b45f05701955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:42.798257 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97338a83-5246-4538-a8a8-b45f05701955 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:42.799017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97338a83-5246-4538-a8a8-b45f05701955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:42.800599 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97338a83-5246-4538-a8a8-b45f05701955 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:42.810985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1003/1991] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:42.815820 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da2ead-2bfd-4109-b565-fbec325ec1e6 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:42.826232 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da2ead-2bfd-4109-b565-fbec325ec1e6 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:42.831831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da646c43-3a69-4d93-8f00-9b443595ecbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:42.832850 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da646c43-3a69-4d93-8f00-9b443595ecbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:42.833131 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da646c43-3a69-4d93-8f00-9b443595ecbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:42.833404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da646c43-3a69-4d93-8f00-9b443595ecbe admin admin] PATH_INFO: `/v3/projects/00ff49d682e645ae8e15b07f6852e3fb/users/f701b5f6632d4da5950da4171e33405b/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:42.849859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da646c43-3a69-4d93-8f00-9b443595ecbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=00ff49d682e645ae8e15b07f6852e3fb, user_id=f701b5f6632d4da5950da4171e33405b, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:42.852521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da646c43-3a69-4d93-8f00-9b443595ecbe admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:42.869733 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 989/1992] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:42 2022] PUT /identity/v3/projects/00ff49d682e645ae8e15b07f6852e3fb/users/f701b5f6632d4da5950da4171e33405b/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:42.880013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad1847-f853-4278-85d4-7f6a0d0f8d65 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:42.880304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad1847-f853-4278-85d4-7f6a0d0f8d65 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:42.880582 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad1847-f853-4278-85d4-7f6a0d0f8d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:42.922837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-fbad1847-f853-4278-85d4-7f6a0d0f8d65 None None] MFA Rules not processed for user `f701b5f6632d4da5950da4171e33405b`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:42.924145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-fbad1847-f853-4278-85d4-7f6a0d0f8d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:42.949308 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:42.992506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1004/1993] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:43.004389 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da646c43-3a69-4d93-8f00-9b443595ecbe admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:43.006249 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-da646c43-3a69-4d93-8f00-9b443595ecbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:43.028794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:43.072381 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da646c43-3a69-4d93-8f00-9b443595ecbe admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:43.076283 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735db104-a78a-42ff-8b2f-d4d7592e7012 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:43.076935 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735db104-a78a-42ff-8b2f-d4d7592e7012 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:43.077218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735db104-a78a-42ff-8b2f-d4d7592e7012 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:43.077494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735db104-a78a-42ff-8b2f-d4d7592e7012 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:43.086379 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735db104-a78a-42ff-8b2f-d4d7592e7012 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:43.087859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735db104-a78a-42ff-8b2f-d4d7592e7012 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:43.096708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 990/1994] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:52.989490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=16 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:53.092615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=17 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:55.991272 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad1847-f853-4278-85d4-7f6a0d0f8d65 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:56.068205 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad1847-f853-4278-85d4-7f6a0d0f8d65 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:56.073353 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc1355-bfc6-49d4-8988-35b40032460b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:56.074267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc1355-bfc6-49d4-8988-35b40032460b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:56.074554 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc1355-bfc6-49d4-8988-35b40032460b admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:56.074832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc1355-bfc6-49d4-8988-35b40032460b admin admin] PATH_INFO: `/v3/users/f701b5f6632d4da5950da4171e33405b` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:56.076127 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc1355-bfc6-49d4-8988-35b40032460b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f701b5f6632d4da5950da4171e33405b)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:56.078090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc1355-bfc6-49d4-8988-35b40032460b admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:56.138424 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-53fc1355-bfc6-49d4-8988-35b40032460b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f701b5f6632d4da5950da4171e33405b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:56.149592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-53fc1355-bfc6-49d4-8988-35b40032460b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f701b5f6632d4da5950da4171e33405b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:56.161797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-53fc1355-bfc6-49d4-8988-35b40032460b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f701b5f6632d4da5950da4171e33405b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:56.162644 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-53fc1355-bfc6-49d4-8988-35b40032460b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f701b5f6632d4da5950da4171e33405b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:56.163368 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-53fc1355-bfc6-49d4-8988-35b40032460b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f701b5f6632d4da5950da4171e33405b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:56.188725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1005/1995] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:55 2022] DELETE /identity/v3/users/f701b5f6632d4da5950da4171e33405b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:56.194548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0936c-15db-4ce1-97f3-a798b60e5a33 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:56.195156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0936c-15db-4ce1-97f3-a798b60e5a33 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:56.195156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0936c-15db-4ce1-97f3-a798b60e5a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:56.213626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-86e0936c-15db-4ce1-97f3-a798b60e5a33 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:56.214862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-86e0936c-15db-4ce1-97f3-a798b60e5a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:56.306977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 991/1996] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:56.320979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc1355-bfc6-49d4-8988-35b40032460b admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:56.322432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-53fc1355-bfc6-49d4-8988-35b40032460b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:56.420387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc1355-bfc6-49d4-8988-35b40032460b admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:56.428035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2db69-10a5-4a07-87b2-ff92f46ffa26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:56.429018 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2db69-10a5-4a07-87b2-ff92f46ffa26 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:56.429347 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2db69-10a5-4a07-87b2-ff92f46ffa26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:56.429746 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2db69-10a5-4a07-87b2-ff92f46ffa26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:56.444200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2db69-10a5-4a07-87b2-ff92f46ffa26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:56.445864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2db69-10a5-4a07-87b2-ff92f46ffa26 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:56.459513 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1006/1997] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:56.792461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0936c-15db-4ce1-97f3-a798b60e5a33 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:56.793907 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-86e0936c-15db-4ce1-97f3-a798b60e5a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:56.811120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0936c-15db-4ce1-97f3-a798b60e5a33 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:56.826778 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de255b-d454-45d7-8dd0-83c37ff0f87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:56.828631 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de255b-d454-45d7-8dd0-83c37ff0f87d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:56.828918 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de255b-d454-45d7-8dd0-83c37ff0f87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:56.829480 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de255b-d454-45d7-8dd0-83c37ff0f87d admin admin] PATH_INFO: `/v3/projects/00ff49d682e645ae8e15b07f6852e3fb` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:56.831870 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de255b-d454-45d7-8dd0-83c37ff0f87d admin admin] RBAC: Authorizing `identity:delete_project(project_id=00ff49d682e645ae8e15b07f6852e3fb)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:56.833821 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de255b-d454-45d7-8dd0-83c37ff0f87d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:56.913036 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f0de255b-d454-45d7-8dd0-83c37ff0f87d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00ff49d682e645ae8e15b07f6852e3fb', 'request_id': 'req-f0de255b-d454-45d7-8dd0-83c37ff0f87d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:56.921463 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f0de255b-d454-45d7-8dd0-83c37ff0f87d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00ff49d682e645ae8e15b07f6852e3fb', 'request_id': 'req-f0de255b-d454-45d7-8dd0-83c37ff0f87d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:56.932254 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f0de255b-d454-45d7-8dd0-83c37ff0f87d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00ff49d682e645ae8e15b07f6852e3fb', 'request_id': 'req-f0de255b-d454-45d7-8dd0-83c37ff0f87d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:56.952910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f0de255b-d454-45d7-8dd0-83c37ff0f87d admin admin] The token cache is being invalidate because project 00ff49d682e645ae8e15b07f6852e3fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:35:56.952910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f0de255b-d454-45d7-8dd0-83c37ff0f87d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:56.953826 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-f0de255b-d454-45d7-8dd0-83c37ff0f87d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:56.957134 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 992/1998] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:35:56 2022] DELETE /identity/v3/projects/00ff49d682e645ae8e15b07f6852e3fb => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:56.963284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c7d31-5849-4ac7-87d5-c898898eddb6 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:56.963607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c7d31-5849-4ac7-87d5-c898898eddb6 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:56.963893 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c7d31-5849-4ac7-87d5-c898898eddb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:56.985091 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-a65c7d31-5849-4ac7-87d5-c898898eddb6 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:56.985091 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-a65c7d31-5849-4ac7-87d5-c898898eddb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:57.004378 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:57.059919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1007/1999] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:57.068540 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f5a7b-12cf-405f-a9e6-22ed525827b6 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:57.069006 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f5a7b-12cf-405f-a9e6-22ed525827b6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:57.069514 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f5a7b-12cf-405f-a9e6-22ed525827b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:57.093632 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-596f5a7b-12cf-405f-a9e6-22ed525827b6 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:57.094739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-596f5a7b-12cf-405f-a9e6-22ed525827b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:57.104298 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 993/2000] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:57.109039 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c7d31-5849-4ac7-87d5-c898898eddb6 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:57.122517 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c7d31-5849-4ac7-87d5-c898898eddb6 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:57.127311 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607b679-73eb-4211-b38e-875a908f79a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:57.128045 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607b679-73eb-4211-b38e-875a908f79a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:57.128320 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607b679-73eb-4211-b38e-875a908f79a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:57.128545 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607b679-73eb-4211-b38e-875a908f79a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:57.129291 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607b679-73eb-4211-b38e-875a908f79a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:35:57.129608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607b679-73eb-4211-b38e-875a908f79a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:57.131257 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607b679-73eb-4211-b38e-875a908f79a7 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:57.146197 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1008/2001] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:35:57.149916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f5a7b-12cf-405f-a9e6-22ed525827b6 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:57.160990 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f5a7b-12cf-405f-a9e6-22ed525827b6 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:57.167312 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4476034-c166-4ed9-a952-33dd83e65cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:57.167765 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4476034-c166-4ed9-a952-33dd83e65cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:57.168055 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4476034-c166-4ed9-a952-33dd83e65cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:57.168332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4476034-c166-4ed9-a952-33dd83e65cb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:57.169101 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4476034-c166-4ed9-a952-33dd83e65cb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:57.171049 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4476034-c166-4ed9-a952-33dd83e65cb2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:57.215285 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 994/2002] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:57 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:57.216955 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607b679-73eb-4211-b38e-875a908f79a7 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:57.226723 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607b679-73eb-4211-b38e-875a908f79a7 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:57.291957 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ff4e9-4f05-4758-9456-b8bd918e0271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:57.292648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ff4e9-4f05-4758-9456-b8bd918e0271 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:57.292935 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ff4e9-4f05-4758-9456-b8bd918e0271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:57.293210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ff4e9-4f05-4758-9456-b8bd918e0271 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:57.293920 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ff4e9-4f05-4758-9456-b8bd918e0271 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:57.295889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ff4e9-4f05-4758-9456-b8bd918e0271 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:57.324469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1009/2003] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:57 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:35:57.325693 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4476034-c166-4ed9-a952-33dd83e65cb2 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:57.336530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4476034-c166-4ed9-a952-33dd83e65cb2 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:57.341651 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62de266-84ea-4bdf-8383-4d7ab05c440a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:57.342328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62de266-84ea-4bdf-8383-4d7ab05c440a admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:57.342611 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62de266-84ea-4bdf-8383-4d7ab05c440a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:57.342893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62de266-84ea-4bdf-8383-4d7ab05c440a admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:57.343803 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62de266-84ea-4bdf-8383-4d7ab05c440a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:57.345569 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62de266-84ea-4bdf-8383-4d7ab05c440a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:57.366624 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 995/2004] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:35:57.368001 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ff4e9-4f05-4758-9456-b8bd918e0271 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:57.378831 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ff4e9-4f05-4758-9456-b8bd918e0271 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:57.383591 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9258755-48c9-46d9-9b30-ce31749d8f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:57.384266 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9258755-48c9-46d9-9b30-ce31749d8f27 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:57.384549 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9258755-48c9-46d9-9b30-ce31749d8f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:57.384827 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9258755-48c9-46d9-9b30-ce31749d8f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:57.385565 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9258755-48c9-46d9-9b30-ce31749d8f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:57.387209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9258755-48c9-46d9-9b30-ce31749d8f27 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:57.402392 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1010/2005] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:57.407236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62de266-84ea-4bdf-8383-4d7ab05c440a admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:57.419534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62de266-84ea-4bdf-8383-4d7ab05c440a admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:57.426353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30fe5b-041b-4d8a-9113-38e314f75e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:57.427482 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30fe5b-041b-4d8a-9113-38e314f75e50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:57.427779 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30fe5b-041b-4d8a-9113-38e314f75e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:57.428060 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30fe5b-041b-4d8a-9113-38e314f75e50 admin admin] PATH_INFO: `/v3/projects/71e451f81c8b4ce1b212e9ea1d34adef/users/a5eb5829add241dcbbdaf42c3b0bfcda/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:57.453005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30fe5b-041b-4d8a-9113-38e314f75e50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e451f81c8b4ce1b212e9ea1d34adef, user_id=a5eb5829add241dcbbdaf42c3b0bfcda, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:57.460379 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30fe5b-041b-4d8a-9113-38e314f75e50 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:57.500684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 996/2006] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:57 2022] PUT /identity/v3/projects/71e451f81c8b4ce1b212e9ea1d34adef/users/a5eb5829add241dcbbdaf42c3b0bfcda/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:57.516802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd71e3-bcdc-475b-9328-e6e22e6dc5c0 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:57.517437 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd71e3-bcdc-475b-9328-e6e22e6dc5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:57.517912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd71e3-bcdc-475b-9328-e6e22e6dc5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:57.552736 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-56bd71e3-bcdc-475b-9328-e6e22e6dc5c0 None None] MFA Rules not processed for user `a5eb5829add241dcbbdaf42c3b0bfcda`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:57.554211 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-56bd71e3-bcdc-475b-9328-e6e22e6dc5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:57.656024 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1011/2007] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:57.662500 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30fe5b-041b-4d8a-9113-38e314f75e50 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:57.687673 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:57.762355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30fe5b-041b-4d8a-9113-38e314f75e50 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:57.770849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2e6e6-e670-4bec-a603-35cedf05f92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:57.772323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2e6e6-e670-4bec-a603-35cedf05f92f admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:57.772833 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2e6e6-e670-4bec-a603-35cedf05f92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:57.773792 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2e6e6-e670-4bec-a603-35cedf05f92f admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:57.774883 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2e6e6-e670-4bec-a603-35cedf05f92f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:57.778700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2e6e6-e670-4bec-a603-35cedf05f92f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:57.818054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 997/2008] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:35:57 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:57.822156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd71e3-bcdc-475b-9328-e6e22e6dc5c0 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:57.840755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd71e3-bcdc-475b-9328-e6e22e6dc5c0 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:57.859811 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:57.918404 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77844a6c-880e-4eba-8dc0-9d3931288a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:57.919268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77844a6c-880e-4eba-8dc0-9d3931288a3e admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:57.919624 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77844a6c-880e-4eba-8dc0-9d3931288a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:57.920156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77844a6c-880e-4eba-8dc0-9d3931288a3e admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:57.922045 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77844a6c-880e-4eba-8dc0-9d3931288a3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:57.925635 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77844a6c-880e-4eba-8dc0-9d3931288a3e admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:57.950237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1012/2009] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:35:57 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:35:57.956093 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2e6e6-e670-4bec-a603-35cedf05f92f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:57.982333 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2e6e6-e670-4bec-a603-35cedf05f92f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:57.986759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c70318-6d98-419a-bbd2-f636edf02d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:57.987716 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c70318-6d98-419a-bbd2-f636edf02d4f admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:57.988243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c70318-6d98-419a-bbd2-f636edf02d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:57.988520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c70318-6d98-419a-bbd2-f636edf02d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:57.989300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c70318-6d98-419a-bbd2-f636edf02d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:57.991900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c70318-6d98-419a-bbd2-f636edf02d4f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:58.004776 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 998/2010] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:35:58.009307 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77844a6c-880e-4eba-8dc0-9d3931288a3e admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:58.026865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77844a6c-880e-4eba-8dc0-9d3931288a3e admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:58.031598 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1ebc6-8607-4427-8c37-94ccd6eca7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:58.032736 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1ebc6-8607-4427-8c37-94ccd6eca7a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:58.033029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1ebc6-8607-4427-8c37-94ccd6eca7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:58.033412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1ebc6-8607-4427-8c37-94ccd6eca7a6 admin admin] PATH_INFO: `/v3/projects/ea3d546a34364a54bca285bd1ab40b9a/users/c6b6fde3660045b1b3564fb36cc4d40a/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:58.054325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1ebc6-8607-4427-8c37-94ccd6eca7a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea3d546a34364a54bca285bd1ab40b9a, user_id=c6b6fde3660045b1b3564fb36cc4d40a, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:58.060623 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1ebc6-8607-4427-8c37-94ccd6eca7a6 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:58.080840 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1013/2011] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:58 2022] PUT /identity/v3/projects/ea3d546a34364a54bca285bd1ab40b9a/users/c6b6fde3660045b1b3564fb36cc4d40a/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:58.085236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c70318-6d98-419a-bbd2-f636edf02d4f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:58.109625 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c70318-6d98-419a-bbd2-f636edf02d4f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:58.211052 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba868de7-ae1c-46ff-8964-2a463f9f089e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:58.212339 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba868de7-ae1c-46ff-8964-2a463f9f089e admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:58.212990 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba868de7-ae1c-46ff-8964-2a463f9f089e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:58.213516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba868de7-ae1c-46ff-8964-2a463f9f089e admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:58.214905 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba868de7-ae1c-46ff-8964-2a463f9f089e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:58.217999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba868de7-ae1c-46ff-8964-2a463f9f089e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:58.238236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 999/2012] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:35:58.244224 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1ebc6-8607-4427-8c37-94ccd6eca7a6 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:58.258524 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1ebc6-8607-4427-8c37-94ccd6eca7a6 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:58.266273 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8baea-35da-4b0b-9ecb-c1b4f3555490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:58.267599 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8baea-35da-4b0b-9ecb-c1b4f3555490 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:58.268516 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8baea-35da-4b0b-9ecb-c1b4f3555490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:58.268864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8baea-35da-4b0b-9ecb-c1b4f3555490 admin admin] PATH_INFO: `/v3/projects/ea3d546a34364a54bca285bd1ab40b9a/users/c6b6fde3660045b1b3564fb36cc4d40a/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:58.271776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8baea-35da-4b0b-9ecb-c1b4f3555490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea3d546a34364a54bca285bd1ab40b9a, user_id=c6b6fde3660045b1b3564fb36cc4d40a, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:58.275720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8baea-35da-4b0b-9ecb-c1b4f3555490 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:58.303620 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1014/2013] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:35:58 2022] PUT /identity/v3/projects/ea3d546a34364a54bca285bd1ab40b9a/users/c6b6fde3660045b1b3564fb36cc4d40a/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:58.317536 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca4c89-ea00-4132-89ec-6498a3b11444 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:58.318238 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca4c89-ea00-4132-89ec-6498a3b11444 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:58.318723 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca4c89-ea00-4132-89ec-6498a3b11444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:58.393894 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-84ca4c89-ea00-4132-89ec-6498a3b11444 None None] MFA Rules not processed for user `c6b6fde3660045b1b3564fb36cc4d40a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:58.395553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-84ca4c89-ea00-4132-89ec-6498a3b11444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:58.499793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1000/2014] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:58.552417 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8baea-35da-4b0b-9ecb-c1b4f3555490 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:58.554658 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e5d8baea-35da-4b0b-9ecb-c1b4f3555490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:58.648725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8baea-35da-4b0b-9ecb-c1b4f3555490 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:58.652445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2eb53-77b8-47f4-89bd-e7d2befd7382 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:58.653112 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2eb53-77b8-47f4-89bd-e7d2befd7382 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:58.653417 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2eb53-77b8-47f4-89bd-e7d2befd7382 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:58.653682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2eb53-77b8-47f4-89bd-e7d2befd7382 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:58.723473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2eb53-77b8-47f4-89bd-e7d2befd7382 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:58.724994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2eb53-77b8-47f4-89bd-e7d2befd7382 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:58.735665 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1015/2015] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:35:59.602718 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca4c89-ea00-4132-89ec-6498a3b11444 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:59.604184 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-84ca4c89-ea00-4132-89ec-6498a3b11444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:59.623468 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:59.688420 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca4c89-ea00-4132-89ec-6498a3b11444 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:59.695876 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537364fc-0693-463e-8943-52660094efbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:59.697477 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537364fc-0693-463e-8943-52660094efbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:59.697477 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537364fc-0693-463e-8943-52660094efbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:59.697477 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537364fc-0693-463e-8943-52660094efbc admin admin] PATH_INFO: `/v3/users/f49f2d3f530f402e96089e89900daaf3` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:59.698571 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537364fc-0693-463e-8943-52660094efbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f49f2d3f530f402e96089e89900daaf3)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:35:59.700573 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537364fc-0693-463e-8943-52660094efbc admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:35:59.762541 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-537364fc-0693-463e-8943-52660094efbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f49f2d3f530f402e96089e89900daaf3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:59.771756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-537364fc-0693-463e-8943-52660094efbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f49f2d3f530f402e96089e89900daaf3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:59.772480 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-537364fc-0693-463e-8943-52660094efbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f49f2d3f530f402e96089e89900daaf3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:59.773099 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-537364fc-0693-463e-8943-52660094efbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f49f2d3f530f402e96089e89900daaf3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:59.785871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-537364fc-0693-463e-8943-52660094efbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f49f2d3f530f402e96089e89900daaf3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:35:59.797929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1001/2016] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:35:59 2022] DELETE /identity/v3/users/f49f2d3f530f402e96089e89900daaf3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:35:59.809127 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d156e-a52f-409c-a586-2cff870565e2 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:59.809969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d156e-a52f-409c-a586-2cff870565e2 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:59.809969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d156e-a52f-409c-a586-2cff870565e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:59.827862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-bc9d156e-a52f-409c-a586-2cff870565e2 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:35:59.829090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-bc9d156e-a52f-409c-a586-2cff870565e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:59.852996 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:35:59.907840 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1016/2017] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:35:59.918364 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537364fc-0693-463e-8943-52660094efbc admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:35:59.919969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-537364fc-0693-463e-8943-52660094efbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:35:59.982062 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537364fc-0693-463e-8943-52660094efbc admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:35:59.985825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5d0a8-e0d5-43ae-b5d3-3760414f2406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:35:59.987002 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5d0a8-e0d5-43ae-b5d3-3760414f2406 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:35:59.987367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5d0a8-e0d5-43ae-b5d3-3760414f2406 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:35:59.987707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5d0a8-e0d5-43ae-b5d3-3760414f2406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:35:59.999246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5d0a8-e0d5-43ae-b5d3-3760414f2406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:00.001018 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5d0a8-e0d5-43ae-b5d3-3760414f2406 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:00.012582 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1002/2018] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:35:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:00.327162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d156e-a52f-409c-a586-2cff870565e2 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:00.328747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-bc9d156e-a52f-409c-a586-2cff870565e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:00.351104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d156e-a52f-409c-a586-2cff870565e2 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:00.358231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f568527-bb69-45b7-90ed-da0f42429171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:00.359703 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f568527-bb69-45b7-90ed-da0f42429171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:00.360204 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f568527-bb69-45b7-90ed-da0f42429171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:00.360714 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f568527-bb69-45b7-90ed-da0f42429171 admin admin] PATH_INFO: `/v3/projects/9ec55138975e48b99ac0f99bf97bb1c0` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:00.363644 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f568527-bb69-45b7-90ed-da0f42429171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ec55138975e48b99ac0f99bf97bb1c0)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:00.367596 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f568527-bb69-45b7-90ed-da0f42429171 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:00.420608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-4f568527-bb69-45b7-90ed-da0f42429171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec55138975e48b99ac0f99bf97bb1c0', 'request_id': 'req-4f568527-bb69-45b7-90ed-da0f42429171'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:00.428876 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-4f568527-bb69-45b7-90ed-da0f42429171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec55138975e48b99ac0f99bf97bb1c0', 'request_id': 'req-4f568527-bb69-45b7-90ed-da0f42429171'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:00.436296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-4f568527-bb69-45b7-90ed-da0f42429171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec55138975e48b99ac0f99bf97bb1c0', 'request_id': 'req-4f568527-bb69-45b7-90ed-da0f42429171'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:00.450446 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-4f568527-bb69-45b7-90ed-da0f42429171 admin admin] The token cache is being invalidate because project 9ec55138975e48b99ac0f99bf97bb1c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:00.450764 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-4f568527-bb69-45b7-90ed-da0f42429171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:00.451446 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-4f568527-bb69-45b7-90ed-da0f42429171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:00.453177 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1017/2019] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:00 2022] DELETE /identity/v3/projects/9ec55138975e48b99ac0f99bf97bb1c0 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:00.457286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5d0a8-e0d5-43ae-b5d3-3760414f2406 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:00.469120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5d0a8-e0d5-43ae-b5d3-3760414f2406 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:00.505667 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:00.530706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c656c7d-6665-42b1-9029-e5364a77d781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:00.531692 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c656c7d-6665-42b1-9029-e5364a77d781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:00.532010 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c656c7d-6665-42b1-9029-e5364a77d781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:00.532240 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c656c7d-6665-42b1-9029-e5364a77d781 admin admin] PATH_INFO: `/v3/users/9c39999ae2244bd2b2b98c5194f823dc` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:00.533980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c656c7d-6665-42b1-9029-e5364a77d781 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c39999ae2244bd2b2b98c5194f823dc)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:00.535951 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c656c7d-6665-42b1-9029-e5364a77d781 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:00.587826 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0c656c7d-6665-42b1-9029-e5364a77d781 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c39999ae2244bd2b2b98c5194f823dc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:00.595550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0c656c7d-6665-42b1-9029-e5364a77d781 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c39999ae2244bd2b2b98c5194f823dc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:00.596679 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0c656c7d-6665-42b1-9029-e5364a77d781 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c39999ae2244bd2b2b98c5194f823dc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:00.597357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0c656c7d-6665-42b1-9029-e5364a77d781 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c39999ae2244bd2b2b98c5194f823dc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:00.611932 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0c656c7d-6665-42b1-9029-e5364a77d781 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c39999ae2244bd2b2b98c5194f823dc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:00.624370 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1003/2020] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:00 2022] DELETE /identity/v3/users/9c39999ae2244bd2b2b98c5194f823dc => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:00.909685 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f568527-bb69-45b7-90ed-da0f42429171 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:00.911345 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4f568527-bb69-45b7-90ed-da0f42429171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:00.927247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f568527-bb69-45b7-90ed-da0f42429171 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:00.984841 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426c020-734f-4bb2-9aae-1ffbd6edb9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:00.985712 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426c020-734f-4bb2-9aae-1ffbd6edb9a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:00.985999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426c020-734f-4bb2-9aae-1ffbd6edb9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:00.986268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426c020-734f-4bb2-9aae-1ffbd6edb9a5 admin admin] PATH_INFO: `/v3/projects/b326d868ab7a43c7a9186dc9d3dbc687` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:00.987519 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6426c020-734f-4bb2-9aae-1ffbd6edb9a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b326d868ab7a43c7a9186dc9d3dbc687)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:00.989465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6426c020-734f-4bb2-9aae-1ffbd6edb9a5 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:01.039719 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6426c020-734f-4bb2-9aae-1ffbd6edb9a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b326d868ab7a43c7a9186dc9d3dbc687', 'request_id': 'req-6426c020-734f-4bb2-9aae-1ffbd6edb9a5'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:01.043640 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:01.049962 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6426c020-734f-4bb2-9aae-1ffbd6edb9a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b326d868ab7a43c7a9186dc9d3dbc687', 'request_id': 'req-6426c020-734f-4bb2-9aae-1ffbd6edb9a5'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:01.059268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6426c020-734f-4bb2-9aae-1ffbd6edb9a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b326d868ab7a43c7a9186dc9d3dbc687', 'request_id': 'req-6426c020-734f-4bb2-9aae-1ffbd6edb9a5'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:01.071543 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6426c020-734f-4bb2-9aae-1ffbd6edb9a5 admin admin] The token cache is being invalidate because project b326d868ab7a43c7a9186dc9d3dbc687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:01.071833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6426c020-734f-4bb2-9aae-1ffbd6edb9a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:01.072528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6426c020-734f-4bb2-9aae-1ffbd6edb9a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:01.074184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1018/2021] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:00 2022] DELETE /identity/v3/projects/b326d868ab7a43c7a9186dc9d3dbc687 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:01.079272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40040ef-14ed-4953-8251-adf602304dcc None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:01.079819 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40040ef-14ed-4953-8251-adf602304dcc None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:01.080927 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40040ef-14ed-4953-8251-adf602304dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:01.109381 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c40040ef-14ed-4953-8251-adf602304dcc None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:01.111427 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c40040ef-14ed-4953-8251-adf602304dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:01.181144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1004/2022] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:36:01.189600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38d39f-dcc1-4598-963c-a93a5805f1ff None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:01.189889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38d39f-dcc1-4598-963c-a93a5805f1ff None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:01.190168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38d39f-dcc1-4598-963c-a93a5805f1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:01.211833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-3e38d39f-dcc1-4598-963c-a93a5805f1ff None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:01.213084 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3e38d39f-dcc1-4598-963c-a93a5805f1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:01.222902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1019/2023] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:01.227443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40040ef-14ed-4953-8251-adf602304dcc None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:01.239118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40040ef-14ed-4953-8251-adf602304dcc None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:01.243989 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eeaa58-abd1-4c04-950f-94f8ef4e9b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:01.244687 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeaa58-abd1-4c04-950f-94f8ef4e9b17 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:01.244953 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeaa58-abd1-4c04-950f-94f8ef4e9b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:01.245232 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeaa58-abd1-4c04-950f-94f8ef4e9b17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:01.246002 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eeaa58-abd1-4c04-950f-94f8ef4e9b17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:36:01.246317 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eeaa58-abd1-4c04-950f-94f8ef4e9b17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:01.247952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eeaa58-abd1-4c04-950f-94f8ef4e9b17 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:01.261707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1005/2024] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:36:01.265351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38d39f-dcc1-4598-963c-a93a5805f1ff None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:01.276709 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38d39f-dcc1-4598-963c-a93a5805f1ff None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:01.281481 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d3f00-fe5c-4eae-9b98-dd0f542eb840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:01.282158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d3f00-fe5c-4eae-9b98-dd0f542eb840 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:01.282439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d3f00-fe5c-4eae-9b98-dd0f542eb840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:01.282709 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d3f00-fe5c-4eae-9b98-dd0f542eb840 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:01.283494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d3f00-fe5c-4eae-9b98-dd0f542eb840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:01.285399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d3f00-fe5c-4eae-9b98-dd0f542eb840 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:01.313806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1020/2025] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:01 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:01.319305 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eeaa58-abd1-4c04-950f-94f8ef4e9b17 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:01.330084 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eeaa58-abd1-4c04-950f-94f8ef4e9b17 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:01.379307 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:01.403811 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cc247-2139-4c76-a458-e0986563137b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:01.404505 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cc247-2139-4c76-a458-e0986563137b admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:01.404846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cc247-2139-4c76-a458-e0986563137b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:01.405128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cc247-2139-4c76-a458-e0986563137b admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:01.405845 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839cc247-2139-4c76-a458-e0986563137b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:01.407836 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839cc247-2139-4c76-a458-e0986563137b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:01.430703 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1006/2026] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:01 2022] POST /identity/v3/users => generated 283 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:01.434906 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d3f00-fe5c-4eae-9b98-dd0f542eb840 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:01.446143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d3f00-fe5c-4eae-9b98-dd0f542eb840 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:01.451024 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d654ff8-52ba-4ac8-80a6-8795a44f5254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:01.451638 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d654ff8-52ba-4ac8-80a6-8795a44f5254 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:01.451949 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d654ff8-52ba-4ac8-80a6-8795a44f5254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:01.452225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d654ff8-52ba-4ac8-80a6-8795a44f5254 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:01.452968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d654ff8-52ba-4ac8-80a6-8795a44f5254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:01.454597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d654ff8-52ba-4ac8-80a6-8795a44f5254 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:01.466764 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1021/2027] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:01.471177 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cc247-2139-4c76-a458-e0986563137b admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:01.483118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cc247-2139-4c76-a458-e0986563137b admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:01.488241 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9b121-cfe3-4600-b316-4ae4079a88e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:01.488936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9b121-cfe3-4600-b316-4ae4079a88e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:01.489225 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9b121-cfe3-4600-b316-4ae4079a88e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:01.489507 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9b121-cfe3-4600-b316-4ae4079a88e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:01.490265 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9b121-cfe3-4600-b316-4ae4079a88e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:01.492049 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9b121-cfe3-4600-b316-4ae4079a88e0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:01.503452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1007/2028] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:01.507985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d654ff8-52ba-4ac8-80a6-8795a44f5254 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:01.518985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d654ff8-52ba-4ac8-80a6-8795a44f5254 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:01.524099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be979d88-9395-4dae-a521-682af8c79e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:01.524986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be979d88-9395-4dae-a521-682af8c79e5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:01.525265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be979d88-9395-4dae-a521-682af8c79e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:01.525539 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be979d88-9395-4dae-a521-682af8c79e5c admin admin] PATH_INFO: `/v3/projects/cbbaccd53acc4f1a98f983815dbdd8e4/users/7d4290571e3f424d86e979fc71e80545/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:01.540145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be979d88-9395-4dae-a521-682af8c79e5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbbaccd53acc4f1a98f983815dbdd8e4, user_id=7d4290571e3f424d86e979fc71e80545, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:01.542791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be979d88-9395-4dae-a521-682af8c79e5c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:01.555883 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1022/2029] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:01 2022] PUT /identity/v3/projects/cbbaccd53acc4f1a98f983815dbdd8e4/users/7d4290571e3f424d86e979fc71e80545/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:01.566877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ff90a-9c46-4e77-9e80-6219a00f89df None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:01.566877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ff90a-9c46-4e77-9e80-6219a00f89df None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:01.567207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ff90a-9c46-4e77-9e80-6219a00f89df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:01.594101 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-fa2ff90a-9c46-4e77-9e80-6219a00f89df None None] MFA Rules not processed for user `7d4290571e3f424d86e979fc71e80545`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:01.595347 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fa2ff90a-9c46-4e77-9e80-6219a00f89df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:01.668868 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1008/2030] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:01.673422 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be979d88-9395-4dae-a521-682af8c79e5c admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:01.747509 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be979d88-9395-4dae-a521-682af8c79e5c admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:01.753135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512ec77-cf8c-494a-a6fd-31759ced0e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:01.753804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512ec77-cf8c-494a-a6fd-31759ced0e80 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:01.754120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512ec77-cf8c-494a-a6fd-31759ced0e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:01.754395 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512ec77-cf8c-494a-a6fd-31759ced0e80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:01.755490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512ec77-cf8c-494a-a6fd-31759ced0e80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:01.757073 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512ec77-cf8c-494a-a6fd-31759ced0e80 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:01.783633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1023/2031] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:01 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:01.787673 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ff90a-9c46-4e77-9e80-6219a00f89df None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:01.799176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ff90a-9c46-4e77-9e80-6219a00f89df None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:01.856757 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487fa7c-7a79-4b12-a089-296b1f4a2879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:01.857434 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487fa7c-7a79-4b12-a089-296b1f4a2879 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:01.857731 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487fa7c-7a79-4b12-a089-296b1f4a2879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:01.857997 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487fa7c-7a79-4b12-a089-296b1f4a2879 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:01.858698 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487fa7c-7a79-4b12-a089-296b1f4a2879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:01.860687 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487fa7c-7a79-4b12-a089-296b1f4a2879 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:01.883138 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1009/2032] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:01 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:01.887266 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512ec77-cf8c-494a-a6fd-31759ced0e80 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:01.898550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512ec77-cf8c-494a-a6fd-31759ced0e80 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:01.903692 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e542a-fe50-4710-beba-688d9e1cd442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:01.904367 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e542a-fe50-4710-beba-688d9e1cd442 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:01.904685 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e542a-fe50-4710-beba-688d9e1cd442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:01.904960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e542a-fe50-4710-beba-688d9e1cd442 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:01.905700 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e542a-fe50-4710-beba-688d9e1cd442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:01.907414 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e542a-fe50-4710-beba-688d9e1cd442 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:01.918694 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1024/2033] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:36:01.923054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487fa7c-7a79-4b12-a089-296b1f4a2879 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:01.934131 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487fa7c-7a79-4b12-a089-296b1f4a2879 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:01.939040 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3585f21-ed88-49df-b57a-2eb2b6dc65fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:01.940465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3585f21-ed88-49df-b57a-2eb2b6dc65fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:01.940753 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3585f21-ed88-49df-b57a-2eb2b6dc65fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:01.941034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3585f21-ed88-49df-b57a-2eb2b6dc65fb admin admin] PATH_INFO: `/v3/projects/9b70af3371e8477a97ed12148d18b5e9/users/8be477acdf2445948606468fc0851f36/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:01.957730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3585f21-ed88-49df-b57a-2eb2b6dc65fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b70af3371e8477a97ed12148d18b5e9, user_id=8be477acdf2445948606468fc0851f36, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:01.960447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3585f21-ed88-49df-b57a-2eb2b6dc65fb admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:01.974815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1010/2034] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:01 2022] PUT /identity/v3/projects/9b70af3371e8477a97ed12148d18b5e9/users/8be477acdf2445948606468fc0851f36/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:01.979263 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e542a-fe50-4710-beba-688d9e1cd442 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:01.990383 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e542a-fe50-4710-beba-688d9e1cd442 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:02.038982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:02.054169 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51cf33-f249-4af8-8bca-0dfd48964fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:02.054854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51cf33-f249-4af8-8bca-0dfd48964fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:02.055167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51cf33-f249-4af8-8bca-0dfd48964fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:02.055442 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51cf33-f249-4af8-8bca-0dfd48964fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:02.056176 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51cf33-f249-4af8-8bca-0dfd48964fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:02.057802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51cf33-f249-4af8-8bca-0dfd48964fb4 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:02.068678 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1025/2035] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:02.073186 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3585f21-ed88-49df-b57a-2eb2b6dc65fb admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:02.084006 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3585f21-ed88-49df-b57a-2eb2b6dc65fb admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:02.089060 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1f308-e884-4818-a510-b2629cb2e992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:02.089978 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1f308-e884-4818-a510-b2629cb2e992 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:02.090273 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1f308-e884-4818-a510-b2629cb2e992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:02.090556 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1f308-e884-4818-a510-b2629cb2e992 admin admin] PATH_INFO: `/v3/projects/9b70af3371e8477a97ed12148d18b5e9/users/8be477acdf2445948606468fc0851f36/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:02.092642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1f308-e884-4818-a510-b2629cb2e992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b70af3371e8477a97ed12148d18b5e9, user_id=8be477acdf2445948606468fc0851f36, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:02.095422 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1f308-e884-4818-a510-b2629cb2e992 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:02.107282 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1011/2036] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:02 2022] PUT /identity/v3/projects/9b70af3371e8477a97ed12148d18b5e9/users/8be477acdf2445948606468fc0851f36/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:02.118879 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabf37c-2a9b-4132-a4a2-46f8aad4c7ed None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:02.119209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabf37c-2a9b-4132-a4a2-46f8aad4c7ed None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:02.119529 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabf37c-2a9b-4132-a4a2-46f8aad4c7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:02.149714 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-dfabf37c-2a9b-4132-a4a2-46f8aad4c7ed None None] MFA Rules not processed for user `8be477acdf2445948606468fc0851f36`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:02.150961 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-dfabf37c-2a9b-4132-a4a2-46f8aad4c7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:02.230468 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1026/2037] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:02.243545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1f308-e884-4818-a510-b2629cb2e992 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:02.246150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-bbe1f308-e884-4818-a510-b2629cb2e992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:02.253626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:02.336575 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1f308-e884-4818-a510-b2629cb2e992 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:02.341218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7711c0-3a65-4be8-bf47-63242c0df0c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:02.342044 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7711c0-3a65-4be8-bf47-63242c0df0c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:02.342473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7711c0-3a65-4be8-bf47-63242c0df0c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:02.342606 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7711c0-3a65-4be8-bf47-63242c0df0c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:02.355255 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7711c0-3a65-4be8-bf47-63242c0df0c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:02.356724 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7711c0-3a65-4be8-bf47-63242c0df0c7 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:02.368119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1012/2038] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:36:02.904652 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabf37c-2a9b-4132-a4a2-46f8aad4c7ed None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:02.975022 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabf37c-2a9b-4132-a4a2-46f8aad4c7ed None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:02.981184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b1acf-3056-42a5-a8a0-4da524e7c5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:02.982334 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b1acf-3056-42a5-a8a0-4da524e7c5f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:02.982777 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b1acf-3056-42a5-a8a0-4da524e7c5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:02.983205 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b1acf-3056-42a5-a8a0-4da524e7c5f1 admin admin] PATH_INFO: `/v3/users/7d4290571e3f424d86e979fc71e80545` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:02.984726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b1acf-3056-42a5-a8a0-4da524e7c5f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d4290571e3f424d86e979fc71e80545)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:02.987394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b1acf-3056-42a5-a8a0-4da524e7c5f1 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:03.058197 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-040b1acf-3056-42a5-a8a0-4da524e7c5f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d4290571e3f424d86e979fc71e80545'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:03.069930 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-040b1acf-3056-42a5-a8a0-4da524e7c5f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d4290571e3f424d86e979fc71e80545'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:03.081116 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-040b1acf-3056-42a5-a8a0-4da524e7c5f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d4290571e3f424d86e979fc71e80545'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:03.082814 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-040b1acf-3056-42a5-a8a0-4da524e7c5f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d4290571e3f424d86e979fc71e80545'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:03.083891 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-040b1acf-3056-42a5-a8a0-4da524e7c5f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d4290571e3f424d86e979fc71e80545'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:03.097044 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:03.118243 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1027/2039] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:02 2022] DELETE /identity/v3/users/7d4290571e3f424d86e979fc71e80545 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:03.135296 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84641c1-f9f8-4d1d-a3d1-a0d003f994b0 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:03.135296 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84641c1-f9f8-4d1d-a3d1-a0d003f994b0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:03.135296 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84641c1-f9f8-4d1d-a3d1-a0d003f994b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:03.167445 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e84641c1-f9f8-4d1d-a3d1-a0d003f994b0 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:03.168795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e84641c1-f9f8-4d1d-a3d1-a0d003f994b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:03.253747 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1013/2040] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:03.266126 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b1acf-3056-42a5-a8a0-4da524e7c5f1 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:03.269023 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-040b1acf-3056-42a5-a8a0-4da524e7c5f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:03.347825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b1acf-3056-42a5-a8a0-4da524e7c5f1 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:03.351840 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87dabe2-083f-4d01-abc1-8ed24f6f3927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:03.352546 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dabe2-083f-4d01-abc1-8ed24f6f3927 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:03.352828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dabe2-083f-4d01-abc1-8ed24f6f3927 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:03.353101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dabe2-083f-4d01-abc1-8ed24f6f3927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:03.363188 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87dabe2-083f-4d01-abc1-8ed24f6f3927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:03.364630 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87dabe2-083f-4d01-abc1-8ed24f6f3927 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:03.376210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1028/2041] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:03.797598 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84641c1-f9f8-4d1d-a3d1-a0d003f994b0 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:03.799277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e84641c1-f9f8-4d1d-a3d1-a0d003f994b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:03.816394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84641c1-f9f8-4d1d-a3d1-a0d003f994b0 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:03.823104 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a41fd5-541b-4835-89ac-64cfc8b30479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:03.824016 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a41fd5-541b-4835-89ac-64cfc8b30479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:03.824494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a41fd5-541b-4835-89ac-64cfc8b30479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:03.824633 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a41fd5-541b-4835-89ac-64cfc8b30479 admin admin] PATH_INFO: `/v3/projects/cbbaccd53acc4f1a98f983815dbdd8e4` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:03.826324 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a41fd5-541b-4835-89ac-64cfc8b30479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbbaccd53acc4f1a98f983815dbdd8e4)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:03.828658 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a41fd5-541b-4835-89ac-64cfc8b30479 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:03.892035 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-89a41fd5-541b-4835-89ac-64cfc8b30479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbbaccd53acc4f1a98f983815dbdd8e4', 'request_id': 'req-89a41fd5-541b-4835-89ac-64cfc8b30479'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:03.899310 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-89a41fd5-541b-4835-89ac-64cfc8b30479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbbaccd53acc4f1a98f983815dbdd8e4', 'request_id': 'req-89a41fd5-541b-4835-89ac-64cfc8b30479'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:03.905377 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-89a41fd5-541b-4835-89ac-64cfc8b30479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbbaccd53acc4f1a98f983815dbdd8e4', 'request_id': 'req-89a41fd5-541b-4835-89ac-64cfc8b30479'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:03.918410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-89a41fd5-541b-4835-89ac-64cfc8b30479 admin admin] The token cache is being invalidate because project cbbaccd53acc4f1a98f983815dbdd8e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:03.918724 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-89a41fd5-541b-4835-89ac-64cfc8b30479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:03.919379 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-89a41fd5-541b-4835-89ac-64cfc8b30479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:03.921214 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1014/2042] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:03 2022] DELETE /identity/v3/projects/cbbaccd53acc4f1a98f983815dbdd8e4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:36:03.925677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87dabe2-083f-4d01-abc1-8ed24f6f3927 service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:03.935466 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87dabe2-083f-4d01-abc1-8ed24f6f3927 service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:04.003068 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915bbfc-df86-4f4d-9449-12f597e5ab0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:04.003968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915bbfc-df86-4f4d-9449-12f597e5ab0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:04.004260 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915bbfc-df86-4f4d-9449-12f597e5ab0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:04.004546 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915bbfc-df86-4f4d-9449-12f597e5ab0f admin admin] PATH_INFO: `/v3/users/8be477acdf2445948606468fc0851f36` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:04.005929 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915bbfc-df86-4f4d-9449-12f597e5ab0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8be477acdf2445948606468fc0851f36)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:04.008044 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915bbfc-df86-4f4d-9449-12f597e5ab0f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:04.069805 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-b915bbfc-df86-4f4d-9449-12f597e5ab0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8be477acdf2445948606468fc0851f36'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:04.081583 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-b915bbfc-df86-4f4d-9449-12f597e5ab0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8be477acdf2445948606468fc0851f36'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:04.090505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-b915bbfc-df86-4f4d-9449-12f597e5ab0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8be477acdf2445948606468fc0851f36'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:04.091651 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-b915bbfc-df86-4f4d-9449-12f597e5ab0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8be477acdf2445948606468fc0851f36'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:04.092725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-b915bbfc-df86-4f4d-9449-12f597e5ab0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8be477acdf2445948606468fc0851f36'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:04.130982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1029/2043] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:03 2022] DELETE /identity/v3/users/8be477acdf2445948606468fc0851f36 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:04.432384 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a41fd5-541b-4835-89ac-64cfc8b30479 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:04.434749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-89a41fd5-541b-4835-89ac-64cfc8b30479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:04.458925 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a41fd5-541b-4835-89ac-64cfc8b30479 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:04.506759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:04.536150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b619c-2f1b-4b88-8982-cd6a1a5881a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:04.537351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b619c-2f1b-4b88-8982-cd6a1a5881a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:04.537853 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b619c-2f1b-4b88-8982-cd6a1a5881a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:04.538345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b619c-2f1b-4b88-8982-cd6a1a5881a5 admin admin] PATH_INFO: `/v3/projects/9b70af3371e8477a97ed12148d18b5e9` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:04.539977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b619c-2f1b-4b88-8982-cd6a1a5881a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b70af3371e8477a97ed12148d18b5e9)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:04.542211 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b619c-2f1b-4b88-8982-cd6a1a5881a5 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:04.589726 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-6a7b619c-2f1b-4b88-8982-cd6a1a5881a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b70af3371e8477a97ed12148d18b5e9', 'request_id': 'req-6a7b619c-2f1b-4b88-8982-cd6a1a5881a5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:04.597193 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-6a7b619c-2f1b-4b88-8982-cd6a1a5881a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b70af3371e8477a97ed12148d18b5e9', 'request_id': 'req-6a7b619c-2f1b-4b88-8982-cd6a1a5881a5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:04.606785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-6a7b619c-2f1b-4b88-8982-cd6a1a5881a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b70af3371e8477a97ed12148d18b5e9', 'request_id': 'req-6a7b619c-2f1b-4b88-8982-cd6a1a5881a5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:04.623799 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-6a7b619c-2f1b-4b88-8982-cd6a1a5881a5 admin admin] The token cache is being invalidate because project 9b70af3371e8477a97ed12148d18b5e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:04.624244 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-6a7b619c-2f1b-4b88-8982-cd6a1a5881a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:04.624979 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-6a7b619c-2f1b-4b88-8982-cd6a1a5881a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:04.627009 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1015/2044] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:04 2022] DELETE /identity/v3/projects/9b70af3371e8477a97ed12148d18b5e9 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:04.631986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05738a6-e526-41ff-8d64-6317dea003e5 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:04.632330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05738a6-e526-41ff-8d64-6317dea003e5 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:04.632682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05738a6-e526-41ff-8d64-6317dea003e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:04.652605 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-a05738a6-e526-41ff-8d64-6317dea003e5 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:04.653824 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-a05738a6-e526-41ff-8d64-6317dea003e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:04.659750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:04.721624 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1030/2045] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:04.729682 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d567a91-dac6-48da-a962-c79a0a5a574d None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:04.729975 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d567a91-dac6-48da-a962-c79a0a5a574d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:04.730254 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d567a91-dac6-48da-a962-c79a0a5a574d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:04.752437 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1d567a91-dac6-48da-a962-c79a0a5a574d None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:04.753669 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1d567a91-dac6-48da-a962-c79a0a5a574d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:04.762550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1016/2046] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:04.767040 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05738a6-e526-41ff-8d64-6317dea003e5 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:04.777853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05738a6-e526-41ff-8d64-6317dea003e5 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:04.782786 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2854ba4-8bbf-4ec7-b29a-877e2930ccb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:04.784119 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2854ba4-8bbf-4ec7-b29a-877e2930ccb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:04.784412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2854ba4-8bbf-4ec7-b29a-877e2930ccb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:04.784602 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2854ba4-8bbf-4ec7-b29a-877e2930ccb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:04.785388 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2854ba4-8bbf-4ec7-b29a-877e2930ccb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:36:04.785629 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2854ba4-8bbf-4ec7-b29a-877e2930ccb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:04.787342 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2854ba4-8bbf-4ec7-b29a-877e2930ccb7 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:04.803606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1031/2047] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:36:04.805229 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d567a91-dac6-48da-a962-c79a0a5a574d None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:04.817120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d567a91-dac6-48da-a962-c79a0a5a574d None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:04.822300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3b2c4-5ddf-4ba1-80d5-8db62eb70e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:04.823113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3b2c4-5ddf-4ba1-80d5-8db62eb70e5f admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:04.823346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3b2c4-5ddf-4ba1-80d5-8db62eb70e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:04.823606 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3b2c4-5ddf-4ba1-80d5-8db62eb70e5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:04.824350 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3b2c4-5ddf-4ba1-80d5-8db62eb70e5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:04.826316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3b2c4-5ddf-4ba1-80d5-8db62eb70e5f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:04.856837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1017/2048] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:04 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:36:04.861011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2854ba4-8bbf-4ec7-b29a-877e2930ccb7 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:04.871789 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2854ba4-8bbf-4ec7-b29a-877e2930ccb7 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:04.943021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7ecda-365e-440a-ac8b-9974cbca03e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:04.943744 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7ecda-365e-440a-ac8b-9974cbca03e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:04.944029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7ecda-365e-440a-ac8b-9974cbca03e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:04.944311 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7ecda-365e-440a-ac8b-9974cbca03e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:04.945046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7ecda-365e-440a-ac8b-9974cbca03e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:04.947036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7ecda-365e-440a-ac8b-9974cbca03e4 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:04.974651 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1032/2049] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:04 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:04.976216 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3b2c4-5ddf-4ba1-80d5-8db62eb70e5f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:04.987723 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3b2c4-5ddf-4ba1-80d5-8db62eb70e5f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:04.992837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a8105-2c36-40a0-9d23-e03922f58a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:04.993523 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a8105-2c36-40a0-9d23-e03922f58a19 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:04.993811 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a8105-2c36-40a0-9d23-e03922f58a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:04.994102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a8105-2c36-40a0-9d23-e03922f58a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:04.994832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a8105-2c36-40a0-9d23-e03922f58a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:04.996493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a8105-2c36-40a0-9d23-e03922f58a19 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:05.008288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1018/2050] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:05.014841 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7ecda-365e-440a-ac8b-9974cbca03e4 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:05.028188 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7ecda-365e-440a-ac8b-9974cbca03e4 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:05.034325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee9cc0-240e-4081-b370-ccac3334a1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:05.035043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee9cc0-240e-4081-b370-ccac3334a1f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:05.035326 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee9cc0-240e-4081-b370-ccac3334a1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:05.035606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee9cc0-240e-4081-b370-ccac3334a1f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:05.036424 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee9cc0-240e-4081-b370-ccac3334a1f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:05.038084 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee9cc0-240e-4081-b370-ccac3334a1f0 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:05.055758 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1033/2051] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:05.060077 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a8105-2c36-40a0-9d23-e03922f58a19 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:05.072968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a8105-2c36-40a0-9d23-e03922f58a19 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:05.077901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cc235-a5c9-49e7-ab69-158e215c1cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:05.078805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cc235-a5c9-49e7-ab69-158e215c1cf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:05.079135 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cc235-a5c9-49e7-ab69-158e215c1cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:05.079419 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cc235-a5c9-49e7-ab69-158e215c1cf5 admin admin] PATH_INFO: `/v3/projects/ceda31c85bf74e5287e32edb89d9736b/users/518efff85e6d4452a9b3edf8ea4bf532/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:05.097014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3cc235-a5c9-49e7-ab69-158e215c1cf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceda31c85bf74e5287e32edb89d9736b, user_id=518efff85e6d4452a9b3edf8ea4bf532, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:05.099725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3cc235-a5c9-49e7-ab69-158e215c1cf5 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:05.115062 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1019/2052] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:05 2022] PUT /identity/v3/projects/ceda31c85bf74e5287e32edb89d9736b/users/518efff85e6d4452a9b3edf8ea4bf532/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:05.125653 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45fd37-e5cd-4e8b-9bf6-2466ca852e21 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:05.125944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45fd37-e5cd-4e8b-9bf6-2466ca852e21 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:05.126222 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45fd37-e5cd-4e8b-9bf6-2466ca852e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:05.166426 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-ba45fd37-e5cd-4e8b-9bf6-2466ca852e21 None None] MFA Rules not processed for user `518efff85e6d4452a9b3edf8ea4bf532`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:05.168707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ba45fd37-e5cd-4e8b-9bf6-2466ca852e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:05.259395 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1034/2053] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:05.264434 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cc235-a5c9-49e7-ab69-158e215c1cf5 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:05.340077 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:05.345172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cc235-a5c9-49e7-ab69-158e215c1cf5 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:05.352329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1bd22-cc2e-4954-87c6-6a89b78f9c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:05.353221 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1bd22-cc2e-4954-87c6-6a89b78f9c84 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:05.353634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1bd22-cc2e-4954-87c6-6a89b78f9c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:05.353996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1bd22-cc2e-4954-87c6-6a89b78f9c84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:05.355338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1bd22-cc2e-4954-87c6-6a89b78f9c84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:05.357851 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1bd22-cc2e-4954-87c6-6a89b78f9c84 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:05.393729 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1020/2054] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:05 2022] POST /identity/v3/projects => generated 389 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:05.398508 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45fd37-e5cd-4e8b-9bf6-2466ca852e21 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:05.410515 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45fd37-e5cd-4e8b-9bf6-2466ca852e21 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:05.414480 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:05.474105 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4fba14-edce-4ad8-9c74-779c0e062cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:05.474796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fba14-edce-4ad8-9c74-779c0e062cad admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:05.475127 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fba14-edce-4ad8-9c74-779c0e062cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:05.475404 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fba14-edce-4ad8-9c74-779c0e062cad admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:05.476100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4fba14-edce-4ad8-9c74-779c0e062cad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:05.478145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4fba14-edce-4ad8-9c74-779c0e062cad admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:05.507491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1035/2055] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:05 2022] POST /identity/v3/users => generated 301 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:36:05.509332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1bd22-cc2e-4954-87c6-6a89b78f9c84 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:05.525172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1bd22-cc2e-4954-87c6-6a89b78f9c84 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:05.532012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c684b1-d138-4c03-86dd-b8f144899156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:05.532783 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c684b1-d138-4c03-86dd-b8f144899156 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:05.534015 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c684b1-d138-4c03-86dd-b8f144899156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:05.534350 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c684b1-d138-4c03-86dd-b8f144899156 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:05.535167 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c684b1-d138-4c03-86dd-b8f144899156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:05.536840 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c684b1-d138-4c03-86dd-b8f144899156 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:05.552396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1021/2056] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:05.553576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4fba14-edce-4ad8-9c74-779c0e062cad admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:05.564636 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4fba14-edce-4ad8-9c74-779c0e062cad admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:05.569545 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbb0e9-654f-4504-981a-a4b27394a734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:05.570424 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbb0e9-654f-4504-981a-a4b27394a734 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:05.570707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbb0e9-654f-4504-981a-a4b27394a734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:05.571024 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbb0e9-654f-4504-981a-a4b27394a734 admin admin] PATH_INFO: `/v3/projects/02e532e216e84314a65274a81e04d2cb/users/1ff2609342bb4331acb0ea0e034e7c45/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:05.587131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbb0e9-654f-4504-981a-a4b27394a734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02e532e216e84314a65274a81e04d2cb, user_id=1ff2609342bb4331acb0ea0e034e7c45, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:05.589766 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbb0e9-654f-4504-981a-a4b27394a734 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:05.603558 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1036/2057] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:05 2022] PUT /identity/v3/projects/02e532e216e84314a65274a81e04d2cb/users/1ff2609342bb4331acb0ea0e034e7c45/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:05.608513 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c684b1-d138-4c03-86dd-b8f144899156 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:05.618873 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c684b1-d138-4c03-86dd-b8f144899156 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:05.657336 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbb0e9-654f-4504-981a-a4b27394a734 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:05.663851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4afbb0e9-654f-4504-981a-a4b27394a734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:05.679369 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87947d55-9cb9-4223-8c8f-516dbb01dd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:05.679967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87947d55-9cb9-4223-8c8f-516dbb01dd16 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:05.680244 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87947d55-9cb9-4223-8c8f-516dbb01dd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:05.680519 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87947d55-9cb9-4223-8c8f-516dbb01dd16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:05.681439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87947d55-9cb9-4223-8c8f-516dbb01dd16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:05.683162 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87947d55-9cb9-4223-8c8f-516dbb01dd16 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:05.695526 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1022/2058] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:05.701120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87947d55-9cb9-4223-8c8f-516dbb01dd16 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:05.710374 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87947d55-9cb9-4223-8c8f-516dbb01dd16 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:05.714993 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b16c1-6e94-4cbd-b6f6-8e6e281902e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:05.715877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b16c1-6e94-4cbd-b6f6-8e6e281902e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:05.716176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b16c1-6e94-4cbd-b6f6-8e6e281902e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:05.716510 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b16c1-6e94-4cbd-b6f6-8e6e281902e8 admin admin] PATH_INFO: `/v3/projects/02e532e216e84314a65274a81e04d2cb/users/1ff2609342bb4331acb0ea0e034e7c45/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:05.718470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b16c1-6e94-4cbd-b6f6-8e6e281902e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02e532e216e84314a65274a81e04d2cb, user_id=1ff2609342bb4331acb0ea0e034e7c45, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:05.721850 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b16c1-6e94-4cbd-b6f6-8e6e281902e8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:05.732518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbb0e9-654f-4504-981a-a4b27394a734 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:05.732933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1023/2059] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:05 2022] PUT /identity/v3/projects/02e532e216e84314a65274a81e04d2cb/users/1ff2609342bb4331acb0ea0e034e7c45/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:36:05.746564 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484778c-ed52-4bca-ac2d-869bc5b07ad0 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:05.746850 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484778c-ed52-4bca-ac2d-869bc5b07ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:05.747188 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484778c-ed52-4bca-ac2d-869bc5b07ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:05.775400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-0484778c-ed52-4bca-ac2d-869bc5b07ad0 None None] MFA Rules not processed for user `1ff2609342bb4331acb0ea0e034e7c45`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:05.776617 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0484778c-ed52-4bca-ac2d-869bc5b07ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:05.800000 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282c200-2666-492c-af1e-6376961a845a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:05.800889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282c200-2666-492c-af1e-6376961a845a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:05.801167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282c200-2666-492c-af1e-6376961a845a admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:05.801440 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282c200-2666-492c-af1e-6376961a845a admin admin] PATH_INFO: `/v3/users/a5eb5829add241dcbbdaf42c3b0bfcda` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:05.802661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282c200-2666-492c-af1e-6376961a845a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5eb5829add241dcbbdaf42c3b0bfcda)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:05.804662 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282c200-2666-492c-af1e-6376961a845a admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:05.827433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:05.860914 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f282c200-2666-492c-af1e-6376961a845a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5eb5829add241dcbbdaf42c3b0bfcda'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:05.864229 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1024/2060] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:05 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:36:05.872617 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f282c200-2666-492c-af1e-6376961a845a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5eb5829add241dcbbdaf42c3b0bfcda'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:05.877351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484778c-ed52-4bca-ac2d-869bc5b07ad0 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:05.879112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0484778c-ed52-4bca-ac2d-869bc5b07ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:05.882454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f282c200-2666-492c-af1e-6376961a845a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5eb5829add241dcbbdaf42c3b0bfcda'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:05.883376 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f282c200-2666-492c-af1e-6376961a845a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5eb5829add241dcbbdaf42c3b0bfcda'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:05.884002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f282c200-2666-492c-af1e-6376961a845a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5eb5829add241dcbbdaf42c3b0bfcda'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:05.903677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1037/2061] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:05 2022] DELETE /identity/v3/users/a5eb5829add241dcbbdaf42c3b0bfcda => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:36:05.907926 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61129f5-9fe6-4846-8e60-ea654aff2342 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:05.908225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61129f5-9fe6-4846-8e60-ea654aff2342 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:05.908507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61129f5-9fe6-4846-8e60-ea654aff2342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:05.924585 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:05.925622 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-d61129f5-9fe6-4846-8e60-ea654aff2342 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:05.927150 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d61129f5-9fe6-4846-8e60-ea654aff2342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:05.943159 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484778c-ed52-4bca-ac2d-869bc5b07ad0 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:05.989730 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1038/2062] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:36:05.994605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b1517-b25e-418e-b753-df7519f83dcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:05.995332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b1517-b25e-418e-b753-df7519f83dcc service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:05.995615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b1517-b25e-418e-b753-df7519f83dcc service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:05.995891 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b1517-b25e-418e-b753-df7519f83dcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:05.998347 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-785b1517-b25e-418e-b753-df7519f83dcc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:06.000307 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61129f5-9fe6-4846-8e60-ea654aff2342 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:06.002154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d61129f5-9fe6-4846-8e60-ea654aff2342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:06.069476 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b1517-b25e-418e-b753-df7519f83dcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:06.069693 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61129f5-9fe6-4846-8e60-ea654aff2342 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:06.071189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b1517-b25e-418e-b753-df7519f83dcc service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:06.073284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe476fb6-dfd7-4641-902e-6f97ac518a51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:06.073940 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe476fb6-dfd7-4641-902e-6f97ac518a51 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:06.074220 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe476fb6-dfd7-4641-902e-6f97ac518a51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:06.074496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe476fb6-dfd7-4641-902e-6f97ac518a51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:06.082402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1025/2063] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:05 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:36:06.085903 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe476fb6-dfd7-4641-902e-6f97ac518a51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:06.087660 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe476fb6-dfd7-4641-902e-6f97ac518a51 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:06.103005 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1039/2064] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:36:06.229693 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b1517-b25e-418e-b753-df7519f83dcc service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:06.232449 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-785b1517-b25e-418e-b753-df7519f83dcc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:06.246767 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b1517-b25e-418e-b753-df7519f83dcc service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:06.254142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9351a0e-5960-4950-b73c-1a1dd75493c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:06.255139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9351a0e-5960-4950-b73c-1a1dd75493c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:06.255597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9351a0e-5960-4950-b73c-1a1dd75493c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:06.256026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9351a0e-5960-4950-b73c-1a1dd75493c2 admin admin] PATH_INFO: `/v3/users/518efff85e6d4452a9b3edf8ea4bf532` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:06.261882 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9351a0e-5960-4950-b73c-1a1dd75493c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=518efff85e6d4452a9b3edf8ea4bf532)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:06.266899 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9351a0e-5960-4950-b73c-1a1dd75493c2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:06.323296 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a9351a0e-5960-4950-b73c-1a1dd75493c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '518efff85e6d4452a9b3edf8ea4bf532'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:06.331994 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a9351a0e-5960-4950-b73c-1a1dd75493c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '518efff85e6d4452a9b3edf8ea4bf532'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:06.332446 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a9351a0e-5960-4950-b73c-1a1dd75493c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '518efff85e6d4452a9b3edf8ea4bf532'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:06.333079 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a9351a0e-5960-4950-b73c-1a1dd75493c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '518efff85e6d4452a9b3edf8ea4bf532'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:06.347153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a9351a0e-5960-4950-b73c-1a1dd75493c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '518efff85e6d4452a9b3edf8ea4bf532'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:06.359899 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1026/2065] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:06 2022] DELETE /identity/v3/users/518efff85e6d4452a9b3edf8ea4bf532 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:06.365009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab89f93-117d-4bdf-bba2-84435162416f None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:06.365491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab89f93-117d-4bdf-bba2-84435162416f None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:06.366070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab89f93-117d-4bdf-bba2-84435162416f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:06.385824 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-cab89f93-117d-4bdf-bba2-84435162416f None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:06.387268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-cab89f93-117d-4bdf-bba2-84435162416f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:06.416106 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9351a0e-5960-4950-b73c-1a1dd75493c2 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:06.417677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a9351a0e-5960-4950-b73c-1a1dd75493c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:06.449810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:06.459864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:06.496398 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1040/2066] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:06.508717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab89f93-117d-4bdf-bba2-84435162416f None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:06.509075 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-cab89f93-117d-4bdf-bba2-84435162416f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:06.514760 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9351a0e-5960-4950-b73c-1a1dd75493c2 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:06.519731 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feea93b-3263-4388-83a3-e9b1f68b4575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:06.520626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feea93b-3263-4388-83a3-e9b1f68b4575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:06.520960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feea93b-3263-4388-83a3-e9b1f68b4575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:06.521523 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feea93b-3263-4388-83a3-e9b1f68b4575 admin admin] PATH_INFO: `/v3/projects/71e451f81c8b4ce1b212e9ea1d34adef` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:06.522747 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feea93b-3263-4388-83a3-e9b1f68b4575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71e451f81c8b4ce1b212e9ea1d34adef)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:06.524875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feea93b-3263-4388-83a3-e9b1f68b4575 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:06.602295 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab89f93-117d-4bdf-bba2-84435162416f None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:06.608854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0feea93b-3263-4388-83a3-e9b1f68b4575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71e451f81c8b4ce1b212e9ea1d34adef', 'request_id': 'req-0feea93b-3263-4388-83a3-e9b1f68b4575'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:06.616843 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0feea93b-3263-4388-83a3-e9b1f68b4575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71e451f81c8b4ce1b212e9ea1d34adef', 'request_id': 'req-0feea93b-3263-4388-83a3-e9b1f68b4575'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:06.632162 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0feea93b-3263-4388-83a3-e9b1f68b4575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71e451f81c8b4ce1b212e9ea1d34adef', 'request_id': 'req-0feea93b-3263-4388-83a3-e9b1f68b4575'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:06.662787 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c638c0-e0bf-434d-8cac-9a8b189af736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:06.663403 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c638c0-e0bf-434d-8cac-9a8b189af736 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:06.663687 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c638c0-e0bf-434d-8cac-9a8b189af736 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:06.663964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c638c0-e0bf-434d-8cac-9a8b189af736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:06.664448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0feea93b-3263-4388-83a3-e9b1f68b4575 admin admin] The token cache is being invalidate because project 71e451f81c8b4ce1b212e9ea1d34adef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:06.667478 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0feea93b-3263-4388-83a3-e9b1f68b4575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:06.668426 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0feea93b-3263-4388-83a3-e9b1f68b4575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:06.673052 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1027/2067] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:06 2022] DELETE /identity/v3/projects/71e451f81c8b4ce1b212e9ea1d34adef => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:36:06.674394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feea93b-3263-4388-83a3-e9b1f68b4575 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:06.676024 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0feea93b-3263-4388-83a3-e9b1f68b4575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:06.748508 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c638c0-e0bf-434d-8cac-9a8b189af736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:06.750235 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c638c0-e0bf-434d-8cac-9a8b189af736 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:06.769711 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1041/2068] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:36:06.824837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feea93b-3263-4388-83a3-e9b1f68b4575 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:06.831261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2e64a-cd20-4332-906f-feff459c822d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:06.832224 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2e64a-cd20-4332-906f-feff459c822d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:06.832626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2e64a-cd20-4332-906f-feff459c822d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:06.833648 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2e64a-cd20-4332-906f-feff459c822d admin admin] PATH_INFO: `/v3/users/c6b6fde3660045b1b3564fb36cc4d40a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:06.835530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a2e64a-cd20-4332-906f-feff459c822d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6b6fde3660045b1b3564fb36cc4d40a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:06.837915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a2e64a-cd20-4332-906f-feff459c822d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:06.896948 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-94a2e64a-cd20-4332-906f-feff459c822d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6b6fde3660045b1b3564fb36cc4d40a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:06.900131 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:06.910012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-94a2e64a-cd20-4332-906f-feff459c822d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6b6fde3660045b1b3564fb36cc4d40a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:06.911572 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-94a2e64a-cd20-4332-906f-feff459c822d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6b6fde3660045b1b3564fb36cc4d40a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:06.913160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-94a2e64a-cd20-4332-906f-feff459c822d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6b6fde3660045b1b3564fb36cc4d40a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:06.931394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-94a2e64a-cd20-4332-906f-feff459c822d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6b6fde3660045b1b3564fb36cc4d40a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:06.951258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1028/2069] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:06 2022] DELETE /identity/v3/users/c6b6fde3660045b1b3564fb36cc4d40a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:07.089987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c638c0-e0bf-434d-8cac-9a8b189af736 service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:07.093660 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-01c638c0-e0bf-434d-8cac-9a8b189af736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:07.107150 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c638c0-e0bf-434d-8cac-9a8b189af736 service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:07.161115 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:07.176761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8da2d-a385-4723-bb4b-e52112378002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:07.177626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8da2d-a385-4723-bb4b-e52112378002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:07.178004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8da2d-a385-4723-bb4b-e52112378002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:07.178194 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8da2d-a385-4723-bb4b-e52112378002 admin admin] PATH_INFO: `/v3/projects/ceda31c85bf74e5287e32edb89d9736b` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:07.179500 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8da2d-a385-4723-bb4b-e52112378002 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceda31c85bf74e5287e32edb89d9736b)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:07.184578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8da2d-a385-4723-bb4b-e52112378002 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:07.233407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-bdc8da2d-a385-4723-bb4b-e52112378002 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceda31c85bf74e5287e32edb89d9736b', 'request_id': 'req-bdc8da2d-a385-4723-bb4b-e52112378002'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:07.241513 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-bdc8da2d-a385-4723-bb4b-e52112378002 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceda31c85bf74e5287e32edb89d9736b', 'request_id': 'req-bdc8da2d-a385-4723-bb4b-e52112378002'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:07.248889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-bdc8da2d-a385-4723-bb4b-e52112378002 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceda31c85bf74e5287e32edb89d9736b', 'request_id': 'req-bdc8da2d-a385-4723-bb4b-e52112378002'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:07.261244 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-bdc8da2d-a385-4723-bb4b-e52112378002 admin admin] The token cache is being invalidate because project ceda31c85bf74e5287e32edb89d9736b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:07.261481 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-bdc8da2d-a385-4723-bb4b-e52112378002 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:07.262118 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-bdc8da2d-a385-4723-bb4b-e52112378002 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:07.263814 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1042/2070] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:07 2022] DELETE /identity/v3/projects/ceda31c85bf74e5287e32edb89d9736b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:07.268361 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2e64a-cd20-4332-906f-feff459c822d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:07.271935 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-94a2e64a-cd20-4332-906f-feff459c822d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:07.289823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2e64a-cd20-4332-906f-feff459c822d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:07.332532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8da2d-a385-4723-bb4b-e52112378002 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:07.334262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-bdc8da2d-a385-4723-bb4b-e52112378002 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:07.382159 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828599fe-5227-4bc5-9389-1ed991e142c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:07.383579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828599fe-5227-4bc5-9389-1ed991e142c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:07.384794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828599fe-5227-4bc5-9389-1ed991e142c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:07.385743 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828599fe-5227-4bc5-9389-1ed991e142c1 admin admin] PATH_INFO: `/v3/users/1ff2609342bb4331acb0ea0e034e7c45` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:07.387254 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828599fe-5227-4bc5-9389-1ed991e142c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ff2609342bb4331acb0ea0e034e7c45)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:07.389255 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828599fe-5227-4bc5-9389-1ed991e142c1 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:07.422152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8da2d-a385-4723-bb4b-e52112378002 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:07.427971 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ce8c0-d139-40cc-8995-eca96f750ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:07.428866 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ce8c0-d139-40cc-8995-eca96f750ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:07.429218 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ce8c0-d139-40cc-8995-eca96f750ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:07.429521 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ce8c0-d139-40cc-8995-eca96f750ec1 admin admin] PATH_INFO: `/v3/projects/ea3d546a34364a54bca285bd1ab40b9a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:07.430734 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ce8c0-d139-40cc-8995-eca96f750ec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea3d546a34364a54bca285bd1ab40b9a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:07.432775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ce8c0-d139-40cc-8995-eca96f750ec1 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:07.456288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-828599fe-5227-4bc5-9389-1ed991e142c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ff2609342bb4331acb0ea0e034e7c45'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:07.465533 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-828599fe-5227-4bc5-9389-1ed991e142c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ff2609342bb4331acb0ea0e034e7c45'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:07.466228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-828599fe-5227-4bc5-9389-1ed991e142c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ff2609342bb4331acb0ea0e034e7c45'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:07.467025 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-828599fe-5227-4bc5-9389-1ed991e142c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ff2609342bb4331acb0ea0e034e7c45'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:07.481176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-828599fe-5227-4bc5-9389-1ed991e142c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ff2609342bb4331acb0ea0e034e7c45'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:07.491828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-c41ce8c0-d139-40cc-8995-eca96f750ec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea3d546a34364a54bca285bd1ab40b9a', 'request_id': 'req-c41ce8c0-d139-40cc-8995-eca96f750ec1'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:07.494700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1029/2071] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:07 2022] DELETE /identity/v3/users/1ff2609342bb4331acb0ea0e034e7c45 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:36:07.505709 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-c41ce8c0-d139-40cc-8995-eca96f750ec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea3d546a34364a54bca285bd1ab40b9a', 'request_id': 'req-c41ce8c0-d139-40cc-8995-eca96f750ec1'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:07.512737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-c41ce8c0-d139-40cc-8995-eca96f750ec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea3d546a34364a54bca285bd1ab40b9a', 'request_id': 'req-c41ce8c0-d139-40cc-8995-eca96f750ec1'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:07.524183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-c41ce8c0-d139-40cc-8995-eca96f750ec1 admin admin] The token cache is being invalidate because project ea3d546a34364a54bca285bd1ab40b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:07.524475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-c41ce8c0-d139-40cc-8995-eca96f750ec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:07.525129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-c41ce8c0-d139-40cc-8995-eca96f750ec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:07.526772 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1043/2072] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:07 2022] DELETE /identity/v3/projects/ea3d546a34364a54bca285bd1ab40b9a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:36:07.531901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453d48d-5100-4a98-b9f8-94a213a22e03 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:07.532363 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453d48d-5100-4a98-b9f8-94a213a22e03 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:07.532527 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453d48d-5100-4a98-b9f8-94a213a22e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:07.551837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8453d48d-5100-4a98-b9f8-94a213a22e03 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:07.553084 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8453d48d-5100-4a98-b9f8-94a213a22e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:07.597420 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:07.619319 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1030/2073] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:36:07.634739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46993e-3390-4512-b320-fedc30aacb87 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:07.634739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46993e-3390-4512-b320-fedc30aacb87 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:07.635315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46993e-3390-4512-b320-fedc30aacb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:07.665027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-ac46993e-3390-4512-b320-fedc30aacb87 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:07.666699 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ac46993e-3390-4512-b320-fedc30aacb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:07.678418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1044/2074] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:07.682736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453d48d-5100-4a98-b9f8-94a213a22e03 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:07.699284 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453d48d-5100-4a98-b9f8-94a213a22e03 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:07.704146 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bf6cf-ec1a-460e-a7f1-796178799d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:07.704945 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bf6cf-ec1a-460e-a7f1-796178799d84 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:07.705795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bf6cf-ec1a-460e-a7f1-796178799d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:07.705960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bf6cf-ec1a-460e-a7f1-796178799d84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:07.706761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597bf6cf-ec1a-460e-a7f1-796178799d84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:36:07.707439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597bf6cf-ec1a-460e-a7f1-796178799d84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:07.708912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597bf6cf-ec1a-460e-a7f1-796178799d84 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:07.726233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1031/2075] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:36:07.727469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46993e-3390-4512-b320-fedc30aacb87 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:07.732744 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:07.740791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46993e-3390-4512-b320-fedc30aacb87 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:07.746607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca279779-f22d-4ca1-b8ed-140196ad654b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:07.747446 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca279779-f22d-4ca1-b8ed-140196ad654b admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:07.747899 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca279779-f22d-4ca1-b8ed-140196ad654b admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:07.748191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca279779-f22d-4ca1-b8ed-140196ad654b admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:07.748887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca279779-f22d-4ca1-b8ed-140196ad654b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:07.751045 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca279779-f22d-4ca1-b8ed-140196ad654b admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:07.782629 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1045/2076] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:07 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:07.784081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bf6cf-ec1a-460e-a7f1-796178799d84 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:07.794396 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bf6cf-ec1a-460e-a7f1-796178799d84 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:07.907755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b92ffc-4cb0-4110-9f99-4fd142b5e821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:07.908898 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b92ffc-4cb0-4110-9f99-4fd142b5e821 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:07.908898 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b92ffc-4cb0-4110-9f99-4fd142b5e821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:07.909176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b92ffc-4cb0-4110-9f99-4fd142b5e821 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:07.909861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b92ffc-4cb0-4110-9f99-4fd142b5e821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:07.912538 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b92ffc-4cb0-4110-9f99-4fd142b5e821 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:07.946654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1032/2077] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:07 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:07.947804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca279779-f22d-4ca1-b8ed-140196ad654b admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:07.962423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca279779-f22d-4ca1-b8ed-140196ad654b admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:07.978577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb275a-434d-495a-82bb-9aa43af594b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:07.979550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb275a-434d-495a-82bb-9aa43af594b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:07.979844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb275a-434d-495a-82bb-9aa43af594b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:07.980125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb275a-434d-495a-82bb-9aa43af594b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:07.981332 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb275a-434d-495a-82bb-9aa43af594b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:07.982985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb275a-434d-495a-82bb-9aa43af594b9 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:07.993383 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b92ffc-4cb0-4110-9f99-4fd142b5e821 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:07.995311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b2b92ffc-4cb0-4110-9f99-4fd142b5e821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:08.006679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1046/2078] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:08.014094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb275a-434d-495a-82bb-9aa43af594b9 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:08.016976 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b92ffc-4cb0-4110-9f99-4fd142b5e821 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:08.029873 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa67ac8-5a52-4c3b-997a-b22b2078e949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:08.030103 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb275a-434d-495a-82bb-9aa43af594b9 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:08.031063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa67ac8-5a52-4c3b-997a-b22b2078e949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:08.031493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa67ac8-5a52-4c3b-997a-b22b2078e949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:08.031842 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa67ac8-5a52-4c3b-997a-b22b2078e949 admin admin] PATH_INFO: `/v3/projects/02e532e216e84314a65274a81e04d2cb` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:08.035145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa67ac8-5a52-4c3b-997a-b22b2078e949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02e532e216e84314a65274a81e04d2cb)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:08.036207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b394b-22e1-41a7-af4a-be91117d644c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:08.037028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b394b-22e1-41a7-af4a-be91117d644c admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:08.037318 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b394b-22e1-41a7-af4a-be91117d644c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:08.037667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b394b-22e1-41a7-af4a-be91117d644c admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:08.037824 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa67ac8-5a52-4c3b-997a-b22b2078e949 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:08.038437 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b394b-22e1-41a7-af4a-be91117d644c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:08.040404 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b394b-22e1-41a7-af4a-be91117d644c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:08.058577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1047/2079] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:08.059832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b394b-22e1-41a7-af4a-be91117d644c admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:08.065161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-2e6b394b-22e1-41a7-af4a-be91117d644c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:08.086065 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b394b-22e1-41a7-af4a-be91117d644c admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:08.099830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b4c0c-fd86-4543-abf4-86baf8dfc323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:08.101938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b4c0c-fd86-4543-abf4-86baf8dfc323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:08.103320 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b4c0c-fd86-4543-abf4-86baf8dfc323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:08.104192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b4c0c-fd86-4543-abf4-86baf8dfc323 admin admin] PATH_INFO: `/v3/users/b5e6ad9771e64da29fed2ed07c64d10e` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:08.109439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b4c0c-fd86-4543-abf4-86baf8dfc323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5e6ad9771e64da29fed2ed07c64d10e)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:08.113997 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b4c0c-fd86-4543-abf4-86baf8dfc323 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:08.116802 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-daa67ac8-5a52-4c3b-997a-b22b2078e949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02e532e216e84314a65274a81e04d2cb', 'request_id': 'req-daa67ac8-5a52-4c3b-997a-b22b2078e949'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:08.124247 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-daa67ac8-5a52-4c3b-997a-b22b2078e949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02e532e216e84314a65274a81e04d2cb', 'request_id': 'req-daa67ac8-5a52-4c3b-997a-b22b2078e949'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:08.131520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-daa67ac8-5a52-4c3b-997a-b22b2078e949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02e532e216e84314a65274a81e04d2cb', 'request_id': 'req-daa67ac8-5a52-4c3b-997a-b22b2078e949'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:08.151014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-daa67ac8-5a52-4c3b-997a-b22b2078e949 admin admin] The token cache is being invalidate because project 02e532e216e84314a65274a81e04d2cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:08.151436 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-daa67ac8-5a52-4c3b-997a-b22b2078e949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:08.152274 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-daa67ac8-5a52-4c3b-997a-b22b2078e949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:08.154228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1033/2080] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:07 2022] DELETE /identity/v3/projects/02e532e216e84314a65274a81e04d2cb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:36:08.155082 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa67ac8-5a52-4c3b-997a-b22b2078e949 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:08.156544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-daa67ac8-5a52-4c3b-997a-b22b2078e949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:08.179731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7b2b4c0c-fd86-4543-abf4-86baf8dfc323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5e6ad9771e64da29fed2ed07c64d10e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:08.189551 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7b2b4c0c-fd86-4543-abf4-86baf8dfc323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5e6ad9771e64da29fed2ed07c64d10e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:08.198038 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7b2b4c0c-fd86-4543-abf4-86baf8dfc323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5e6ad9771e64da29fed2ed07c64d10e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:08.198818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7b2b4c0c-fd86-4543-abf4-86baf8dfc323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5e6ad9771e64da29fed2ed07c64d10e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:08.199484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7b2b4c0c-fd86-4543-abf4-86baf8dfc323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5e6ad9771e64da29fed2ed07c64d10e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:08.220255 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:08.224346 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1048/2081] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:08 2022] DELETE /identity/v3/users/b5e6ad9771e64da29fed2ed07c64d10e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:08.226093 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc9454-32ae-402e-b75f-ea88b8762361 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:08.226490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc9454-32ae-402e-b75f-ea88b8762361 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:08.227527 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc9454-32ae-402e-b75f-ea88b8762361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:08.243977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa67ac8-5a52-4c3b-997a-b22b2078e949 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:08.248362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-5ccc9454-32ae-402e-b75f-ea88b8762361 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:08.249595 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-5ccc9454-32ae-402e-b75f-ea88b8762361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:08.319819 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4220886-335b-45e9-a610-7a25c78dc561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:08.321017 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4220886-335b-45e9-a610-7a25c78dc561 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:08.321572 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4220886-335b-45e9-a610-7a25c78dc561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:08.321686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1049/2082] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:08.321924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4220886-335b-45e9-a610-7a25c78dc561 admin admin] PATH_INFO: `/v3/projects/22784ca7f8ba4bebb836c35bd673f0fc/users/188ff2b1d0b542a8923d5b097a7062b1/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:08.324362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51d372-3ec7-41d8-9f6a-8fe472532e8a None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:08.326729 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51d372-3ec7-41d8-9f6a-8fe472532e8a None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:08.327633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51d372-3ec7-41d8-9f6a-8fe472532e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:08.346695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4220886-335b-45e9-a610-7a25c78dc561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22784ca7f8ba4bebb836c35bd673f0fc, user_id=188ff2b1d0b542a8923d5b097a7062b1, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:08.350690 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4220886-335b-45e9-a610-7a25c78dc561 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:08.351051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-7c51d372-3ec7-41d8-9f6a-8fe472532e8a None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:08.352381 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-7c51d372-3ec7-41d8-9f6a-8fe472532e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:08.362420 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1050/2083] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:08.363781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d852a-9971-4041-9e29-066ba6dae200 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:08.364074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d852a-9971-4041-9e29-066ba6dae200 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:08.364357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d852a-9971-4041-9e29-066ba6dae200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:08.378564 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1034/2084] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:08 2022] PUT /identity/v3/projects/22784ca7f8ba4bebb836c35bd673f0fc/users/188ff2b1d0b542a8923d5b097a7062b1/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:08.379696 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4220886-335b-45e9-a610-7a25c78dc561 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:08.390455 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a4220886-335b-45e9-a610-7a25c78dc561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:08.391928 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-073d852a-9971-4041-9e29-066ba6dae200 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:08.393791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-073d852a-9971-4041-9e29-066ba6dae200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:08.420557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:08.472339 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4220886-335b-45e9-a610-7a25c78dc561 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:08.479685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d1a0e-9053-4f54-9758-02f2fd34cf4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:08.480666 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d1a0e-9053-4f54-9758-02f2fd34cf4d service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:08.481117 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d1a0e-9053-4f54-9758-02f2fd34cf4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:08.482011 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d1a0e-9053-4f54-9758-02f2fd34cf4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:08.502473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1051/2085] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:08.508337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbf672-4f4e-4866-9029-1ea2dd86e905 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:08.508712 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbf672-4f4e-4866-9029-1ea2dd86e905 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:08.509064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbf672-4f4e-4866-9029-1ea2dd86e905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:08.541281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-25dbf672-4f4e-4866-9029-1ea2dd86e905 None None] MFA Rules not processed for user `188ff2b1d0b542a8923d5b097a7062b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:08.542618 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-25dbf672-4f4e-4866-9029-1ea2dd86e905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:08.573596 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d1a0e-9053-4f54-9758-02f2fd34cf4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:08.578569 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d1a0e-9053-4f54-9758-02f2fd34cf4d service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:08.599971 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1035/2086] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:08.609494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d1a0e-9053-4f54-9758-02f2fd34cf4d service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:08.633786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d1a0e-9053-4f54-9758-02f2fd34cf4d service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:08.643163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a046e-38d0-489b-b7bc-a7feae471660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:08.644291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a046e-38d0-489b-b7bc-a7feae471660 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:08.645112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a046e-38d0-489b-b7bc-a7feae471660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:08.645626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a046e-38d0-489b-b7bc-a7feae471660 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:08.647744 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a046e-38d0-489b-b7bc-a7feae471660 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:36:08.648340 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a046e-38d0-489b-b7bc-a7feae471660 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:08.650612 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a046e-38d0-489b-b7bc-a7feae471660 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:08.671781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1052/2087] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:08.674549 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1036/2088] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:36:08.683512 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbf672-4f4e-4866-9029-1ea2dd86e905 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:08.684117 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a046e-38d0-489b-b7bc-a7feae471660 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:08.686568 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-25dbf672-4f4e-4866-9029-1ea2dd86e905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:08.697517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a046e-38d0-489b-b7bc-a7feae471660 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:08.707292 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b3b3d-f176-4153-82df-75e2135f25ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:08.708022 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b3b3d-f176-4153-82df-75e2135f25ee admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:08.708315 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b3b3d-f176-4153-82df-75e2135f25ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:08.708598 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b3b3d-f176-4153-82df-75e2135f25ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:08.709307 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b3b3d-f176-4153-82df-75e2135f25ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:08.709984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbf672-4f4e-4866-9029-1ea2dd86e905 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:08.711824 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b3b3d-f176-4153-82df-75e2135f25ee admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:08.718075 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd160fd8-840f-4061-9032-582f891f483d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:08.719009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd160fd8-840f-4061-9032-582f891f483d admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:08.719521 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd160fd8-840f-4061-9032-582f891f483d admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:08.720072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd160fd8-840f-4061-9032-582f891f483d admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:08.722001 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd160fd8-840f-4061-9032-582f891f483d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:08.725220 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd160fd8-840f-4061-9032-582f891f483d admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:08.733182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:08.762417 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1053/2089] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:08 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:08.766451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd160fd8-840f-4061-9032-582f891f483d admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:08.780621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1037/2090] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:08 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:08.784270 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b3b3d-f176-4153-82df-75e2135f25ee admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:08.882869 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd160fd8-840f-4061-9032-582f891f483d admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:08.928350 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b3b3d-f176-4153-82df-75e2135f25ee admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:08.934108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcf84e-9258-477a-bf0b-9f8e7df9cac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:08.934898 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcf84e-9258-477a-bf0b-9f8e7df9cac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:08.935209 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcf84e-9258-477a-bf0b-9f8e7df9cac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:08.935487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcf84e-9258-477a-bf0b-9f8e7df9cac1 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:08.936354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcf84e-9258-477a-bf0b-9f8e7df9cac1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:08.938535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcf84e-9258-477a-bf0b-9f8e7df9cac1 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:08.954613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:08.975639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1038/2091] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:08 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:36:08.980580 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcf84e-9258-477a-bf0b-9f8e7df9cac1 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:08.995794 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97849097-76ce-4d7c-9684-e5801b471a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:08.996530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcf84e-9258-477a-bf0b-9f8e7df9cac1 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:08.996706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97849097-76ce-4d7c-9684-e5801b471a95 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:08.997086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97849097-76ce-4d7c-9684-e5801b471a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:08.997437 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97849097-76ce-4d7c-9684-e5801b471a95 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:08.998221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97849097-76ce-4d7c-9684-e5801b471a95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:09.001221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97849097-76ce-4d7c-9684-e5801b471a95 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:09.004278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645da25-7746-4761-931c-4b7c854a3a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:09.005386 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645da25-7746-4761-931c-4b7c854a3a22 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.005823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645da25-7746-4761-931c-4b7c854a3a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.006254 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645da25-7746-4761-931c-4b7c854a3a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.007424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645da25-7746-4761-931c-4b7c854a3a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:09.009804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645da25-7746-4761-931c-4b7c854a3a22 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:09.023647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1039/2092] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:09.024752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645da25-7746-4761-931c-4b7c854a3a22 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:09.027340 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4645da25-7746-4761-931c-4b7c854a3a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:09.046927 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645da25-7746-4761-931c-4b7c854a3a22 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:09.048569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1054/2093] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:08 2022] POST /identity/v3/users => generated 293 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:09.049256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97849097-76ce-4d7c-9684-e5801b471a95 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:09.054841 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0e33d-ad73-402d-ba36-b21d5f7118f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:09.056412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0e33d-ad73-402d-ba36-b21d5f7118f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.057174 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0e33d-ad73-402d-ba36-b21d5f7118f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.057963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0e33d-ad73-402d-ba36-b21d5f7118f0 admin admin] PATH_INFO: `/v3/projects/218c95949b8843cbbac812c44625e517` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.059278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0e33d-ad73-402d-ba36-b21d5f7118f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=218c95949b8843cbbac812c44625e517)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:09.066757 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97849097-76ce-4d7c-9684-e5801b471a95 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:09.068786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0e33d-ad73-402d-ba36-b21d5f7118f0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:09.079115 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e3c11-729d-4e4b-a0e1-6fbd551c5006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:09.080024 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e3c11-729d-4e4b-a0e1-6fbd551c5006 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.080915 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e3c11-729d-4e4b-a0e1-6fbd551c5006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.081315 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e3c11-729d-4e4b-a0e1-6fbd551c5006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.082187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e3c11-729d-4e4b-a0e1-6fbd551c5006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:09.084191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e3c11-729d-4e4b-a0e1-6fbd551c5006 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:09.102875 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1055/2094] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:09.102875 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e3c11-729d-4e4b-a0e1-6fbd551c5006 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:09.117097 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e3c11-729d-4e4b-a0e1-6fbd551c5006 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:09.124140 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e3a9e-0885-4e23-af88-00c75a05bbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:09.125054 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e3a9e-0885-4e23-af88-00c75a05bbec admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.125597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e3a9e-0885-4e23-af88-00c75a05bbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.126125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e3a9e-0885-4e23-af88-00c75a05bbec admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.133715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e3a9e-0885-4e23-af88-00c75a05bbec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:09.135567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e3a9e-0885-4e23-af88-00c75a05bbec admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:09.154077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1056/2095] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:09.154077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e3a9e-0885-4e23-af88-00c75a05bbec admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:09.169035 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c4f0e33d-ad73-402d-ba36-b21d5f7118f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '218c95949b8843cbbac812c44625e517', 'request_id': 'req-c4f0e33d-ad73-402d-ba36-b21d5f7118f0'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:09.174086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e3a9e-0885-4e23-af88-00c75a05bbec admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:09.178664 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c4f0e33d-ad73-402d-ba36-b21d5f7118f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '218c95949b8843cbbac812c44625e517', 'request_id': 'req-c4f0e33d-ad73-402d-ba36-b21d5f7118f0'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:09.186721 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c4f0e33d-ad73-402d-ba36-b21d5f7118f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '218c95949b8843cbbac812c44625e517', 'request_id': 'req-c4f0e33d-ad73-402d-ba36-b21d5f7118f0'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:09.207796 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c4f0e33d-ad73-402d-ba36-b21d5f7118f0 admin admin] The token cache is being invalidate because project 218c95949b8843cbbac812c44625e517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:09.208126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c4f0e33d-ad73-402d-ba36-b21d5f7118f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:09.208879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c4f0e33d-ad73-402d-ba36-b21d5f7118f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:09.210735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1040/2096] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:09 2022] DELETE /identity/v3/projects/218c95949b8843cbbac812c44625e517 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:09.211376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0e33d-ad73-402d-ba36-b21d5f7118f0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:09.212861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c4f0e33d-ad73-402d-ba36-b21d5f7118f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:09.254229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4972587-f72d-4450-96c6-cfa64aa9b40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:09.255410 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4972587-f72d-4450-96c6-cfa64aa9b40f admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.255795 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4972587-f72d-4450-96c6-cfa64aa9b40f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.256170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4972587-f72d-4450-96c6-cfa64aa9b40f admin admin] PATH_INFO: `/v3/projects/4381071944e443358be2865a989a7f8c/users/2976ae7b389d450492aa9382b2446f5f/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.276880 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4972587-f72d-4450-96c6-cfa64aa9b40f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4381071944e443358be2865a989a7f8c, user_id=2976ae7b389d450492aa9382b2446f5f, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:09.279589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4972587-f72d-4450-96c6-cfa64aa9b40f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:09.292523 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:09.294198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1057/2097] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:09 2022] PUT /identity/v3/projects/4381071944e443358be2865a989a7f8c/users/2976ae7b389d450492aa9382b2446f5f/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:09.304997 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790d377-33c8-4eb3-9e9e-e68b400bb8a6 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.304997 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790d377-33c8-4eb3-9e9e-e68b400bb8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.305480 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790d377-33c8-4eb3-9e9e-e68b400bb8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.308984 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0e33d-ad73-402d-ba36-b21d5f7118f0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:09.323027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-9790d377-33c8-4eb3-9e9e-e68b400bb8a6 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:09.324288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9790d377-33c8-4eb3-9e9e-e68b400bb8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:09.365622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3977fe-cc66-49a6-ad02-2e2af607d323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:09.366300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3977fe-cc66-49a6-ad02-2e2af607d323 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.366586 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3977fe-cc66-49a6-ad02-2e2af607d323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.366866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3977fe-cc66-49a6-ad02-2e2af607d323 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.367639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3977fe-cc66-49a6-ad02-2e2af607d323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:09.369264 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3977fe-cc66-49a6-ad02-2e2af607d323 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:09.380586 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1041/2098] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:09.382729 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3eb0d-6b35-474d-8f8a-d4c9d345778c None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.383552 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3eb0d-6b35-474d-8f8a-d4c9d345778c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.384068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3eb0d-6b35-474d-8f8a-d4c9d345778c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.386083 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1058/2099] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:09.386906 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790d377-33c8-4eb3-9e9e-e68b400bb8a6 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:09.398303 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790d377-33c8-4eb3-9e9e-e68b400bb8a6 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:09.403365 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd438d2a-68dd-4807-bf5f-add0ae4f99c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:09.404308 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd438d2a-68dd-4807-bf5f-add0ae4f99c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.404599 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd438d2a-68dd-4807-bf5f-add0ae4f99c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.404991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd438d2a-68dd-4807-bf5f-add0ae4f99c2 admin admin] PATH_INFO: `/v3/projects/14699e92ffcd4de7b4ed6b48fc6e29f0/users/b30f6783fb644c8aa395ca3c5f43f0e4/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.415589 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-61d3eb0d-6b35-474d-8f8a-d4c9d345778c None None] MFA Rules not processed for user `2976ae7b389d450492aa9382b2446f5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:09.416810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-61d3eb0d-6b35-474d-8f8a-d4c9d345778c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:09.420408 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd438d2a-68dd-4807-bf5f-add0ae4f99c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14699e92ffcd4de7b4ed6b48fc6e29f0, user_id=b30f6783fb644c8aa395ca3c5f43f0e4, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:09.423178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd438d2a-68dd-4807-bf5f-add0ae4f99c2 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:09.436194 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1059/2100] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:09 2022] PUT /identity/v3/projects/14699e92ffcd4de7b4ed6b48fc6e29f0/users/b30f6783fb644c8aa395ca3c5f43f0e4/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:09.437378 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60ac56-19b6-4b83-b4c6-e55d9abaa97b None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.437683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60ac56-19b6-4b83-b4c6-e55d9abaa97b None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.437940 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60ac56-19b6-4b83-b4c6-e55d9abaa97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.450093 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:09.457532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-ce60ac56-19b6-4b83-b4c6-e55d9abaa97b None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:09.458908 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ce60ac56-19b6-4b83-b4c6-e55d9abaa97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:09.509265 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1042/2101] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:09.510699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae4251-78fd-40f9-8d79-62dab5e4fc2b None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.511023 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae4251-78fd-40f9-8d79-62dab5e4fc2b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.511312 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae4251-78fd-40f9-8d79-62dab5e4fc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.536402 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1060/2102] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:09.537414 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60ac56-19b6-4b83-b4c6-e55d9abaa97b None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:09.541244 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ce60ac56-19b6-4b83-b4c6-e55d9abaa97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:09.559516 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60ac56-19b6-4b83-b4c6-e55d9abaa97b None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:09.576051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580f53f-e60c-4159-9f9a-95031c3e980d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:09.576051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580f53f-e60c-4159-9f9a-95031c3e980d admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.576348 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e5ae4251-78fd-40f9-8d79-62dab5e4fc2b None None] MFA Rules not processed for user `b30f6783fb644c8aa395ca3c5f43f0e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:09.576804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580f53f-e60c-4159-9f9a-95031c3e980d admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.576804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580f53f-e60c-4159-9f9a-95031c3e980d admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.576804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580f53f-e60c-4159-9f9a-95031c3e980d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:09.577042 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e5ae4251-78fd-40f9-8d79-62dab5e4fc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:09.577136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580f53f-e60c-4159-9f9a-95031c3e980d admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:09.603690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1061/2103] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:09 2022] POST /identity/v3/projects => generated 387 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:09.604618 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580f53f-e60c-4159-9f9a-95031c3e980d admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:09.646508 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1043/2104] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:09.651012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae4251-78fd-40f9-8d79-62dab5e4fc2b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:09.661519 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae4251-78fd-40f9-8d79-62dab5e4fc2b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:09.674715 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:09.685405 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580f53f-e60c-4159-9f9a-95031c3e980d admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:09.690984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edb6c1-9164-4d61-bb37-b4d9c12ce4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:09.691826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edb6c1-9164-4d61-bb37-b4d9c12ce4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.692325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edb6c1-9164-4d61-bb37-b4d9c12ce4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.692713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edb6c1-9164-4d61-bb37-b4d9c12ce4a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.693453 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3edb6c1-9164-4d61-bb37-b4d9c12ce4a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:36:09.693760 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3edb6c1-9164-4d61-bb37-b4d9c12ce4a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:09.695391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3edb6c1-9164-4d61-bb37-b4d9c12ce4a7 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:09.710791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1062/2105] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:36:09.710791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edb6c1-9164-4d61-bb37-b4d9c12ce4a7 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:09.726057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edb6c1-9164-4d61-bb37-b4d9c12ce4a7 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:09.731832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e8ecf-65ce-4ae0-a7fd-7aca0441ef0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:09.732737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e8ecf-65ce-4ae0-a7fd-7aca0441ef0c admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.733104 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e8ecf-65ce-4ae0-a7fd-7aca0441ef0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.733405 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e8ecf-65ce-4ae0-a7fd-7aca0441ef0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.733970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e8ecf-65ce-4ae0-a7fd-7aca0441ef0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:09.735833 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6c478-7c31-4674-a33b-f7c78551dc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:09.736224 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e8ecf-65ce-4ae0-a7fd-7aca0441ef0c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:09.737060 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6c478-7c31-4674-a33b-f7c78551dc6b admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.737410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6c478-7c31-4674-a33b-f7c78551dc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.737941 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6c478-7c31-4674-a33b-f7c78551dc6b admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.738844 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6c478-7c31-4674-a33b-f7c78551dc6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:09.741147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6c478-7c31-4674-a33b-f7c78551dc6b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:09.773281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1063/2106] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:09 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:09.773281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e8ecf-65ce-4ae0-a7fd-7aca0441ef0c admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:09.777540 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1044/2107] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:09 2022] POST /identity/v3/users => generated 300 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:09.778964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6c478-7c31-4674-a33b-f7c78551dc6b admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:09.796208 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6c478-7c31-4674-a33b-f7c78551dc6b admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:09.845713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:09.850440 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e8ecf-65ce-4ae0-a7fd-7aca0441ef0c admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:09.858256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab31ed-959f-4537-9a5b-8910133fae80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:09.858962 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab31ed-959f-4537-9a5b-8910133fae80 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.859633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab31ed-959f-4537-9a5b-8910133fae80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.859907 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab31ed-959f-4537-9a5b-8910133fae80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.861592 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab31ed-959f-4537-9a5b-8910133fae80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:09.865062 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab31ed-959f-4537-9a5b-8910133fae80 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:09.872974 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cf98b-c491-4493-ad50-4f9908514f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:09.873831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cf98b-c491-4493-ad50-4f9908514f48 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.874215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cf98b-c491-4493-ad50-4f9908514f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.874593 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cf98b-c491-4493-ad50-4f9908514f48 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.875448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17cf98b-c491-4493-ad50-4f9908514f48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:09.878141 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17cf98b-c491-4493-ad50-4f9908514f48 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:09.899745 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1064/2108] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:09 2022] POST /identity/v3/projects => generated 377 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:09.902871 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab31ed-959f-4537-9a5b-8910133fae80 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:09.907177 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1045/2109] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:09 2022] POST /identity/v3/users => generated 298 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:09.907861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cf98b-c491-4493-ad50-4f9908514f48 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:09.919322 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cf98b-c491-4493-ad50-4f9908514f48 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:09.969639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab31ed-959f-4537-9a5b-8910133fae80 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:09.974257 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079fafd-323d-4d9a-af7d-2da3351cf316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:09.975045 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079fafd-323d-4d9a-af7d-2da3351cf316 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.975429 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079fafd-323d-4d9a-af7d-2da3351cf316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.975706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079fafd-323d-4d9a-af7d-2da3351cf316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.976430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079fafd-323d-4d9a-af7d-2da3351cf316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:09.976624 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aaf004-f092-406d-b133-4958137f82c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:09.977269 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aaf004-f092-406d-b133-4958137f82c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:09.977551 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aaf004-f092-406d-b133-4958137f82c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:09.977825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aaf004-f092-406d-b133-4958137f82c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:09.978013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079fafd-323d-4d9a-af7d-2da3351cf316 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:09.978515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aaf004-f092-406d-b133-4958137f82c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:09.980451 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aaf004-f092-406d-b133-4958137f82c9 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:09.988692 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1065/2110] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:09.989547 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079fafd-323d-4d9a-af7d-2da3351cf316 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:10.000229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079fafd-323d-4d9a-af7d-2da3351cf316 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:10.008303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1046/2111] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:09 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:10.008451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bf0e7-ae40-43a3-82a3-946df462089b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:10.008549 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aaf004-f092-406d-b133-4958137f82c9 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:10.008648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bf0e7-ae40-43a3-82a3-946df462089b admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.008648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bf0e7-ae40-43a3-82a3-946df462089b admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.008648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bf0e7-ae40-43a3-82a3-946df462089b admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.008648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342bf0e7-ae40-43a3-82a3-946df462089b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:10.009324 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342bf0e7-ae40-43a3-82a3-946df462089b admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:10.016351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aaf004-f092-406d-b133-4958137f82c9 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:10.023145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1066/2112] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:10.023145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bf0e7-ae40-43a3-82a3-946df462089b admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:10.023353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a6a7a-37c3-4864-b0de-198c43f893c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:10.023353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a6a7a-37c3-4864-b0de-198c43f893c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.023353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a6a7a-37c3-4864-b0de-198c43f893c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.023353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a6a7a-37c3-4864-b0de-198c43f893c8 admin admin] PATH_INFO: `/v3/projects/161a5c1b39d14778bc0a27e8c3fa45a6/users/f9867d136fdf405a83b34cf45ed61e09/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.030703 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bf0e7-ae40-43a3-82a3-946df462089b admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:10.035794 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d981133-598c-4267-82e9-31c0e1027085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:10.036451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d981133-598c-4267-82e9-31c0e1027085 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.036734 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d981133-598c-4267-82e9-31c0e1027085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.037013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d981133-598c-4267-82e9-31c0e1027085 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.037864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d981133-598c-4267-82e9-31c0e1027085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:10.039249 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a6a7a-37c3-4864-b0de-198c43f893c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=161a5c1b39d14778bc0a27e8c3fa45a6, user_id=f9867d136fdf405a83b34cf45ed61e09, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:10.041189 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d981133-598c-4267-82e9-31c0e1027085 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:10.042768 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a6a7a-37c3-4864-b0de-198c43f893c8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:10.057401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:10.057608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1067/2113] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:10.058466 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d981133-598c-4267-82e9-31c0e1027085 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:10.063901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1047/2114] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:10 2022] PUT /identity/v3/projects/161a5c1b39d14778bc0a27e8c3fa45a6/users/f9867d136fdf405a83b34cf45ed61e09/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:10.065979 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a6a7a-37c3-4864-b0de-198c43f893c8 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:10.070834 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d981133-598c-4267-82e9-31c0e1027085 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:10.134180 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cc567-1b10-4069-b970-02645be87b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:10.135123 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cc567-1b10-4069-b970-02645be87b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.135410 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cc567-1b10-4069-b970-02645be87b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.135692 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cc567-1b10-4069-b970-02645be87b63 admin admin] PATH_INFO: `/v3/projects/43ebe22b20c94f49bb43c4689891fbd6/users/748ba04206184146a9ae623d5fec2121/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.139441 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a6a7a-37c3-4864-b0de-198c43f893c8 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:10.144320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e632455-c7d0-4ea9-893c-eb69bc2cad1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:10.145070 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e632455-c7d0-4ea9-893c-eb69bc2cad1a admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.145346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e632455-c7d0-4ea9-893c-eb69bc2cad1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.145618 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e632455-c7d0-4ea9-893c-eb69bc2cad1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.146607 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e632455-c7d0-4ea9-893c-eb69bc2cad1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:10.148251 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e632455-c7d0-4ea9-893c-eb69bc2cad1a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:10.151381 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9cc567-1b10-4069-b970-02645be87b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43ebe22b20c94f49bb43c4689891fbd6, user_id=748ba04206184146a9ae623d5fec2121, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:10.154052 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9cc567-1b10-4069-b970-02645be87b63 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:10.158873 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1048/2115] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:10.160048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e632455-c7d0-4ea9-893c-eb69bc2cad1a admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:10.171597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e632455-c7d0-4ea9-893c-eb69bc2cad1a admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:10.172186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1068/2116] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:10 2022] PUT /identity/v3/projects/43ebe22b20c94f49bb43c4689891fbd6/users/748ba04206184146a9ae623d5fec2121/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:10.174082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cc567-1b10-4069-b970-02645be87b63 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:10.242669 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:10.249126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0023cf-4bda-4001-ab1a-a71e0763a1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:10.249961 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0023cf-4bda-4001-ab1a-a71e0763a1ed admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.250348 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0023cf-4bda-4001-ab1a-a71e0763a1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.250730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0023cf-4bda-4001-ab1a-a71e0763a1ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.252265 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0023cf-4bda-4001-ab1a-a71e0763a1ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:10.253932 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0023cf-4bda-4001-ab1a-a71e0763a1ed admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:10.266093 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1049/2117] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:10.271104 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0023cf-4bda-4001-ab1a-a71e0763a1ed admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:10.273244 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cc567-1b10-4069-b970-02645be87b63 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:10.279886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cc4ae-96de-4c9a-b554-9719013d6fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:10.280831 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cc4ae-96de-4c9a-b554-9719013d6fc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.281263 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cc4ae-96de-4c9a-b554-9719013d6fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.281562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cc4ae-96de-4c9a-b554-9719013d6fc3 admin admin] PATH_INFO: `/v3/projects/b13d332f75044502bc92a00ca050a7f9/users/bb72c2809fe94d008e2acc5fc24c2316/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.286609 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0023cf-4bda-4001-ab1a-a71e0763a1ed admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:10.292625 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562f62d-00f0-47ec-9857-8db0fb00ae3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:10.293304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562f62d-00f0-47ec-9857-8db0fb00ae3b admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.293586 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562f62d-00f0-47ec-9857-8db0fb00ae3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.293865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562f62d-00f0-47ec-9857-8db0fb00ae3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.294693 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562f62d-00f0-47ec-9857-8db0fb00ae3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:10.296394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562f62d-00f0-47ec-9857-8db0fb00ae3b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:10.301523 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cc4ae-96de-4c9a-b554-9719013d6fc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b13d332f75044502bc92a00ca050a7f9, user_id=bb72c2809fe94d008e2acc5fc24c2316, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:10.305035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cc4ae-96de-4c9a-b554-9719013d6fc3 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:10.310218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1050/2118] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:10.311144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562f62d-00f0-47ec-9857-8db0fb00ae3b admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:10.321476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1069/2119] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:10 2022] PUT /identity/v3/projects/b13d332f75044502bc92a00ca050a7f9/users/bb72c2809fe94d008e2acc5fc24c2316/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:10.322361 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562f62d-00f0-47ec-9857-8db0fb00ae3b admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:10.323021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cc4ae-96de-4c9a-b554-9719013d6fc3 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:10.391619 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a2cc5-a1fd-455b-8985-5fe2a0a04ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:10.392768 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a2cc5-a1fd-455b-8985-5fe2a0a04ca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.393056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a2cc5-a1fd-455b-8985-5fe2a0a04ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.393370 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a2cc5-a1fd-455b-8985-5fe2a0a04ca6 admin admin] PATH_INFO: `/v3/projects/161a5c1b39d14778bc0a27e8c3fa45a6/users/f9867d136fdf405a83b34cf45ed61e09/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.395720 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a2cc5-a1fd-455b-8985-5fe2a0a04ca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=161a5c1b39d14778bc0a27e8c3fa45a6, user_id=f9867d136fdf405a83b34cf45ed61e09, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:10.399189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a2cc5-a1fd-455b-8985-5fe2a0a04ca6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:10.401448 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cc4ae-96de-4c9a-b554-9719013d6fc3 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:10.406247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291001ae-e96b-4bdc-aeb3-dad716b17607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:10.407179 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291001ae-e96b-4bdc-aeb3-dad716b17607 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.407469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291001ae-e96b-4bdc-aeb3-dad716b17607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.407751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291001ae-e96b-4bdc-aeb3-dad716b17607 admin admin] PATH_INFO: `/v3/projects/43ebe22b20c94f49bb43c4689891fbd6/users/748ba04206184146a9ae623d5fec2121/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.410320 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291001ae-e96b-4bdc-aeb3-dad716b17607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43ebe22b20c94f49bb43c4689891fbd6, user_id=748ba04206184146a9ae623d5fec2121, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:10.410604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1051/2120] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:10 2022] PUT /identity/v3/projects/161a5c1b39d14778bc0a27e8c3fa45a6/users/f9867d136fdf405a83b34cf45ed61e09/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:10.411990 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098187c-750c-438e-9d5f-b0f6387c4459 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.412279 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098187c-750c-438e-9d5f-b0f6387c4459 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.412564 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098187c-750c-438e-9d5f-b0f6387c4459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.413096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291001ae-e96b-4bdc-aeb3-dad716b17607 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:10.424771 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1070/2121] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:10 2022] PUT /identity/v3/projects/43ebe22b20c94f49bb43c4689891fbd6/users/748ba04206184146a9ae623d5fec2121/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:10.432597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3cb70-d0cf-4a03-9b31-d0af2d251f1a None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.432979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3cb70-d0cf-4a03-9b31-d0af2d251f1a None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.433334 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3cb70-d0cf-4a03-9b31-d0af2d251f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.450307 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8098187c-750c-438e-9d5f-b0f6387c4459 None None] MFA Rules not processed for user `bb72c2809fe94d008e2acc5fc24c2316`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:10.451604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8098187c-750c-438e-9d5f-b0f6387c4459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:10.455885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:10.462236 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-c3a3cb70-d0cf-4a03-9b31-d0af2d251f1a None None] MFA Rules not processed for user `f9867d136fdf405a83b34cf45ed61e09`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:10.463610 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-c3a3cb70-d0cf-4a03-9b31-d0af2d251f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:10.528710 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1052/2122] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:10.529831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9587d-d89b-4a3e-8473-c6bb98118ce3 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.530120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9587d-d89b-4a3e-8473-c6bb98118ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.530398 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9587d-d89b-4a3e-8473-c6bb98118ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.552052 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1071/2123] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:10.552454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3cb70-d0cf-4a03-9b31-d0af2d251f1a None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:10.560032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8ad9587d-d89b-4a3e-8473-c6bb98118ce3 None None] MFA Rules not processed for user `748ba04206184146a9ae623d5fec2121`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:10.561231 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8ad9587d-d89b-4a3e-8473-c6bb98118ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:10.600028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:10.624746 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3cb70-d0cf-4a03-9b31-d0af2d251f1a None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:10.637286 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafcfcd-7616-4f9d-b7d2-7a1de4fc5332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:10.637961 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafcfcd-7616-4f9d-b7d2-7a1de4fc5332 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.638244 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafcfcd-7616-4f9d-b7d2-7a1de4fc5332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.638521 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafcfcd-7616-4f9d-b7d2-7a1de4fc5332 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.639267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafcfcd-7616-4f9d-b7d2-7a1de4fc5332 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:10.641219 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafcfcd-7616-4f9d-b7d2-7a1de4fc5332 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:10.652047 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1053/2124] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:10.652993 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9587d-d89b-4a3e-8473-c6bb98118ce3 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:10.654648 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8ad9587d-d89b-4a3e-8473-c6bb98118ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:10.672811 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1072/2125] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:10 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:10.677758 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafcfcd-7616-4f9d-b7d2-7a1de4fc5332 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:10.720496 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9587d-d89b-4a3e-8473-c6bb98118ce3 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:10.744020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafcfcd-7616-4f9d-b7d2-7a1de4fc5332 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:10.749092 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f19d5-aa20-4b5e-acbe-65eb02880410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:10.749881 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f19d5-aa20-4b5e-acbe-65eb02880410 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.750167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f19d5-aa20-4b5e-acbe-65eb02880410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.750441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f19d5-aa20-4b5e-acbe-65eb02880410 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.751182 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f19d5-aa20-4b5e-acbe-65eb02880410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:10.753283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f19d5-aa20-4b5e-acbe-65eb02880410 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:10.767280 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:10.773770 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84795444-1824-4b75-a839-6ec42cd75039 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:10.774428 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84795444-1824-4b75-a839-6ec42cd75039 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.774724 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84795444-1824-4b75-a839-6ec42cd75039 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.775003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84795444-1824-4b75-a839-6ec42cd75039 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.781630 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1073/2126] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:10 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:36:10.781630 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f19d5-aa20-4b5e-acbe-65eb02880410 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:10.782927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-a92f19d5-aa20-4b5e-acbe-65eb02880410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:10.795777 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f19d5-aa20-4b5e-acbe-65eb02880410 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:10.800414 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32608cc-fe39-42a3-bbbd-e43424f516be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:10.801550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32608cc-fe39-42a3-bbbd-e43424f516be service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.801965 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32608cc-fe39-42a3-bbbd-e43424f516be service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.802352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32608cc-fe39-42a3-bbbd-e43424f516be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.804631 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-c32608cc-fe39-42a3-bbbd-e43424f516be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:10.837550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84795444-1824-4b75-a839-6ec42cd75039 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:10.839152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84795444-1824-4b75-a839-6ec42cd75039 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:10.849355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1054/2127] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:10.850298 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84795444-1824-4b75-a839-6ec42cd75039 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:10.861412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84795444-1824-4b75-a839-6ec42cd75039 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:10.868225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32608cc-fe39-42a3-bbbd-e43424f516be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:10.868408 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4acf10-f66e-4505-98c1-e1e6edf9c2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:10.869159 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4acf10-f66e-4505-98c1-e1e6edf9c2ca admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.869485 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4acf10-f66e-4505-98c1-e1e6edf9c2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.869772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4acf10-f66e-4505-98c1-e1e6edf9c2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.870198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32608cc-fe39-42a3-bbbd-e43424f516be service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:10.870545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4acf10-f66e-4505-98c1-e1e6edf9c2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:10.872389 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4acf10-f66e-4505-98c1-e1e6edf9c2ca admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:10.881421 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1074/2128] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:10.890207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1055/2129] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:10.894299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32608cc-fe39-42a3-bbbd-e43424f516be service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:10.905359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32608cc-fe39-42a3-bbbd-e43424f516be service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:10.911806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62344b2-a191-412c-9e17-79f71e547457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:10.912506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62344b2-a191-412c-9e17-79f71e547457 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.912875 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62344b2-a191-412c-9e17-79f71e547457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.913052 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62344b2-a191-412c-9e17-79f71e547457 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.913846 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62344b2-a191-412c-9e17-79f71e547457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:10.915404 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62344b2-a191-412c-9e17-79f71e547457 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:10.927470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1075/2130] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:10.932065 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4acf10-f66e-4505-98c1-e1e6edf9c2ca admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:10.951780 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4acf10-f66e-4505-98c1-e1e6edf9c2ca admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:10.951780 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74fb90-68a9-44fe-88b5-cf2c0a9040ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:10.951780 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74fb90-68a9-44fe-88b5-cf2c0a9040ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:10.952048 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74fb90-68a9-44fe-88b5-cf2c0a9040ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:10.952224 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74fb90-68a9-44fe-88b5-cf2c0a9040ba admin admin] PATH_INFO: `/v3/projects/a2f412b417e14db885dfe3cfa6059876/users/f3413c3709be4c22823c88cf2f208820/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:10.972781 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74fb90-68a9-44fe-88b5-cf2c0a9040ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2f412b417e14db885dfe3cfa6059876, user_id=f3413c3709be4c22823c88cf2f208820, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:10.976330 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74fb90-68a9-44fe-88b5-cf2c0a9040ba admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:10.996338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1056/2131] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:10 2022] PUT /identity/v3/projects/a2f412b417e14db885dfe3cfa6059876/users/f3413c3709be4c22823c88cf2f208820/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:11.014845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fee32-0028-41a8-9d91-e8974342e8bf None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:11.014845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fee32-0028-41a8-9d91-e8974342e8bf None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:11.014845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fee32-0028-41a8-9d91-e8974342e8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:11.088051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-c28fee32-0028-41a8-9d91-e8974342e8bf None None] MFA Rules not processed for user `f3413c3709be4c22823c88cf2f208820`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:11.089475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-c28fee32-0028-41a8-9d91-e8974342e8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:11.150366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:11.228735 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1076/2132] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:11.229494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74fb90-68a9-44fe-88b5-cf2c0a9040ba admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:11.319759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74fb90-68a9-44fe-88b5-cf2c0a9040ba admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:11.325557 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02050735-3b94-4dcc-a8cd-4d1825aad707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:11.326303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02050735-3b94-4dcc-a8cd-4d1825aad707 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:11.326754 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02050735-3b94-4dcc-a8cd-4d1825aad707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:11.327054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02050735-3b94-4dcc-a8cd-4d1825aad707 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:11.327759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02050735-3b94-4dcc-a8cd-4d1825aad707 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:11.329996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02050735-3b94-4dcc-a8cd-4d1825aad707 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:11.384996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1057/2133] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:11 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:11.393939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fee32-0028-41a8-9d91-e8974342e8bf None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:11.407656 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fee32-0028-41a8-9d91-e8974342e8bf None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:11.480197 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4390a6-b4da-41ac-8d84-e71f73a69eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:11.480948 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4390a6-b4da-41ac-8d84-e71f73a69eed admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:11.481338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4390a6-b4da-41ac-8d84-e71f73a69eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:11.481725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4390a6-b4da-41ac-8d84-e71f73a69eed admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:11.482530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4390a6-b4da-41ac-8d84-e71f73a69eed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:11.484941 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4390a6-b4da-41ac-8d84-e71f73a69eed admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:11.508761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1077/2134] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:11 2022] POST /identity/v3/users => generated 295 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:11.513628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02050735-3b94-4dcc-a8cd-4d1825aad707 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:11.529403 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02050735-3b94-4dcc-a8cd-4d1825aad707 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:11.535466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d18ec-1e01-4db3-9b10-76b1e72805bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:11.536155 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d18ec-1e01-4db3-9b10-76b1e72805bf admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:11.536442 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d18ec-1e01-4db3-9b10-76b1e72805bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:11.536716 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d18ec-1e01-4db3-9b10-76b1e72805bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:11.537458 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d18ec-1e01-4db3-9b10-76b1e72805bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:11.539098 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d18ec-1e01-4db3-9b10-76b1e72805bf admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:11.550643 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1058/2135] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:36:11.554711 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4390a6-b4da-41ac-8d84-e71f73a69eed admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:11.565797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4390a6-b4da-41ac-8d84-e71f73a69eed admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:11.571122 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe35d8-f466-4c3f-901b-f93b7c7333db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:11.572065 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe35d8-f466-4c3f-901b-f93b7c7333db admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:11.572353 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe35d8-f466-4c3f-901b-f93b7c7333db admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:11.572716 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe35d8-f466-4c3f-901b-f93b7c7333db admin admin] PATH_INFO: `/v3/projects/44da29c85fa341debb198a0b2a47f79e/users/fe41d0ab2d8b4f8d95d8be995833ba0a/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:11.589725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe35d8-f466-4c3f-901b-f93b7c7333db admin admin] RBAC: Authorizing `identity:create_grant(project_id=44da29c85fa341debb198a0b2a47f79e, user_id=fe41d0ab2d8b4f8d95d8be995833ba0a, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:11.592445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe35d8-f466-4c3f-901b-f93b7c7333db admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:11.609061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1078/2136] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:11 2022] PUT /identity/v3/projects/44da29c85fa341debb198a0b2a47f79e/users/fe41d0ab2d8b4f8d95d8be995833ba0a/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:11.613715 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d18ec-1e01-4db3-9b10-76b1e72805bf admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:11.624182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d18ec-1e01-4db3-9b10-76b1e72805bf admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:11.674429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:11.707866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af60b58-0fdd-4a0e-8b30-3bbb0564079f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:11.708551 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af60b58-0fdd-4a0e-8b30-3bbb0564079f admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:11.708836 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af60b58-0fdd-4a0e-8b30-3bbb0564079f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:11.709118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af60b58-0fdd-4a0e-8b30-3bbb0564079f admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:11.709859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af60b58-0fdd-4a0e-8b30-3bbb0564079f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:11.711614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af60b58-0fdd-4a0e-8b30-3bbb0564079f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:11.722586 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1059/2137] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:11.727501 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe35d8-f466-4c3f-901b-f93b7c7333db admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:11.739354 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe35d8-f466-4c3f-901b-f93b7c7333db admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:11.744679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae387ef-35d1-4044-9988-08d42fb76c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:11.745687 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae387ef-35d1-4044-9988-08d42fb76c9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:11.746068 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae387ef-35d1-4044-9988-08d42fb76c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:11.746417 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae387ef-35d1-4044-9988-08d42fb76c9f admin admin] PATH_INFO: `/v3/projects/44da29c85fa341debb198a0b2a47f79e/users/fe41d0ab2d8b4f8d95d8be995833ba0a/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:11.748546 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae387ef-35d1-4044-9988-08d42fb76c9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=44da29c85fa341debb198a0b2a47f79e, user_id=fe41d0ab2d8b4f8d95d8be995833ba0a, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:11.751283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae387ef-35d1-4044-9988-08d42fb76c9f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:11.763957 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1079/2138] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:11 2022] PUT /identity/v3/projects/44da29c85fa341debb198a0b2a47f79e/users/fe41d0ab2d8b4f8d95d8be995833ba0a/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:11.775475 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551b85f-9af7-4902-9376-90724bd91b34 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:11.775791 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551b85f-9af7-4902-9376-90724bd91b34 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:11.776115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551b85f-9af7-4902-9376-90724bd91b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:11.802408 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae387ef-35d1-4044-9988-08d42fb76c9f admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:11.808543 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d551b85f-9af7-4902-9376-90724bd91b34 None None] MFA Rules not processed for user `fe41d0ab2d8b4f8d95d8be995833ba0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:11.810327 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d551b85f-9af7-4902-9376-90724bd91b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:11.872414 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae387ef-35d1-4044-9988-08d42fb76c9f admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:11.878832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472be8b-f5c2-45bf-844f-dea68466dea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:11.880345 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472be8b-f5c2-45bf-844f-dea68466dea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:11.880797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472be8b-f5c2-45bf-844f-dea68466dea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:11.881112 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472be8b-f5c2-45bf-844f-dea68466dea8 admin admin] PATH_INFO: `/v3/users/2976ae7b389d450492aa9382b2446f5f` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:11.883304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472be8b-f5c2-45bf-844f-dea68466dea8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2976ae7b389d450492aa9382b2446f5f)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:11.885683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472be8b-f5c2-45bf-844f-dea68466dea8 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:11.893910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1060/2139] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:11.907526 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551b85f-9af7-4902-9376-90724bd91b34 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:11.922317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:11.949066 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-b472be8b-f5c2-45bf-844f-dea68466dea8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2976ae7b389d450492aa9382b2446f5f'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:11.958179 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-b472be8b-f5c2-45bf-844f-dea68466dea8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2976ae7b389d450492aa9382b2446f5f'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:11.966424 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-b472be8b-f5c2-45bf-844f-dea68466dea8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2976ae7b389d450492aa9382b2446f5f'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:11.967776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-b472be8b-f5c2-45bf-844f-dea68466dea8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2976ae7b389d450492aa9382b2446f5f'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:11.968467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-b472be8b-f5c2-45bf-844f-dea68466dea8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2976ae7b389d450492aa9382b2446f5f'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:11.976764 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551b85f-9af7-4902-9376-90724bd91b34 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:11.981183 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39325e3-e964-4f47-bb55-0a8543ecf092 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:11.981870 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39325e3-e964-4f47-bb55-0a8543ecf092 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:11.982153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39325e3-e964-4f47-bb55-0a8543ecf092 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:11.982424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39325e3-e964-4f47-bb55-0a8543ecf092 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:11.984305 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b39325e3-e964-4f47-bb55-0a8543ecf092 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:11.992648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1080/2140] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:11 2022] DELETE /identity/v3/users/2976ae7b389d450492aa9382b2446f5f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:11.997876 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70790c0-c89f-4308-a5d3-7ae0857420b4 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:11.998261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70790c0-c89f-4308-a5d3-7ae0857420b4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:11.998621 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70790c0-c89f-4308-a5d3-7ae0857420b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:12.018489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-b70790c0-c89f-4308-a5d3-7ae0857420b4 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:12.019750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b70790c0-c89f-4308-a5d3-7ae0857420b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:12.050697 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39325e3-e964-4f47-bb55-0a8543ecf092 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:12.052207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39325e3-e964-4f47-bb55-0a8543ecf092 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:12.061587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1061/2141] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:36:12.082518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1081/2142] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:36:12.092482 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39325e3-e964-4f47-bb55-0a8543ecf092 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:12.094039 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b39325e3-e964-4f47-bb55-0a8543ecf092 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:12.108935 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:12.162062 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39325e3-e964-4f47-bb55-0a8543ecf092 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:12.226876 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60ec83-aa36-4e99-a355-b5ed5d7fc0a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:12.227652 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60ec83-aa36-4e99-a355-b5ed5d7fc0a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:12.227947 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60ec83-aa36-4e99-a355-b5ed5d7fc0a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:12.228236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60ec83-aa36-4e99-a355-b5ed5d7fc0a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:12.240562 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60ec83-aa36-4e99-a355-b5ed5d7fc0a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:12.242491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60ec83-aa36-4e99-a355-b5ed5d7fc0a6 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:12.255027 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1062/2143] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:12.707948 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70790c0-c89f-4308-a5d3-7ae0857420b4 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:12.710699 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b70790c0-c89f-4308-a5d3-7ae0857420b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:12.745175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70790c0-c89f-4308-a5d3-7ae0857420b4 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:12.769791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4dba6-f294-4b4f-9c68-927ec52a4e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:12.773703 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4dba6-f294-4b4f-9c68-927ec52a4e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:12.777642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4dba6-f294-4b4f-9c68-927ec52a4e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:12.778475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4dba6-f294-4b4f-9c68-927ec52a4e93 admin admin] PATH_INFO: `/v3/projects/4381071944e443358be2865a989a7f8c` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:12.785695 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4dba6-f294-4b4f-9c68-927ec52a4e93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4381071944e443358be2865a989a7f8c)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:12.792308 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4dba6-f294-4b4f-9c68-927ec52a4e93 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:12.804490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60ec83-aa36-4e99-a355-b5ed5d7fc0a6 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:12.806924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5e60ec83-aa36-4e99-a355-b5ed5d7fc0a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:12.893040 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-72a4dba6-f294-4b4f-9c68-927ec52a4e93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4381071944e443358be2865a989a7f8c', 'request_id': 'req-72a4dba6-f294-4b4f-9c68-927ec52a4e93'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:12.906450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-72a4dba6-f294-4b4f-9c68-927ec52a4e93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4381071944e443358be2865a989a7f8c', 'request_id': 'req-72a4dba6-f294-4b4f-9c68-927ec52a4e93'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:12.912951 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60ec83-aa36-4e99-a355-b5ed5d7fc0a6 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:12.914239 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-72a4dba6-f294-4b4f-9c68-927ec52a4e93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4381071944e443358be2865a989a7f8c', 'request_id': 'req-72a4dba6-f294-4b4f-9c68-927ec52a4e93'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:12.931474 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-72a4dba6-f294-4b4f-9c68-927ec52a4e93 admin admin] The token cache is being invalidate because project 4381071944e443358be2865a989a7f8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:12.931914 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-72a4dba6-f294-4b4f-9c68-927ec52a4e93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:12.932870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-72a4dba6-f294-4b4f-9c68-927ec52a4e93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:12.935027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1082/2144] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:12 2022] DELETE /identity/v3/projects/4381071944e443358be2865a989a7f8c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:12.941399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4dba6-f294-4b4f-9c68-927ec52a4e93 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:12.943255 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-72a4dba6-f294-4b4f-9c68-927ec52a4e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:12.988296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:13.018200 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed3309-6840-494a-a0ec-e21eaf463a4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:13.019041 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed3309-6840-494a-a0ec-e21eaf463a4a service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:13.019345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed3309-6840-494a-a0ec-e21eaf463a4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:13.019717 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed3309-6840-494a-a0ec-e21eaf463a4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:13.023376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-47ed3309-6840-494a-a0ec-e21eaf463a4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:13.042424 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4dba6-f294-4b4f-9c68-927ec52a4e93 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:13.049486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fdefb9-2926-4007-a754-73f21018af02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:13.050399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fdefb9-2926-4007-a754-73f21018af02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:13.051200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fdefb9-2926-4007-a754-73f21018af02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:13.051512 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fdefb9-2926-4007-a754-73f21018af02 admin admin] PATH_INFO: `/v3/users/f9867d136fdf405a83b34cf45ed61e09` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:13.052859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fdefb9-2926-4007-a754-73f21018af02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9867d136fdf405a83b34cf45ed61e09)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:13.054804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fdefb9-2926-4007-a754-73f21018af02 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:13.078059 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:13.104583 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed3309-6840-494a-a0ec-e21eaf463a4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:13.106349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed3309-6840-494a-a0ec-e21eaf463a4a service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:13.116106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-70fdefb9-2926-4007-a754-73f21018af02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9867d136fdf405a83b34cf45ed61e09'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:13.117650 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1063/2145] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:36:13.128861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-70fdefb9-2926-4007-a754-73f21018af02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9867d136fdf405a83b34cf45ed61e09'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:13.137848 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-70fdefb9-2926-4007-a754-73f21018af02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9867d136fdf405a83b34cf45ed61e09'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:13.138855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-70fdefb9-2926-4007-a754-73f21018af02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9867d136fdf405a83b34cf45ed61e09'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:13.141850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-70fdefb9-2926-4007-a754-73f21018af02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9867d136fdf405a83b34cf45ed61e09'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:13.164808 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1083/2146] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:12 2022] DELETE /identity/v3/users/f9867d136fdf405a83b34cf45ed61e09 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:13.675167 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed3309-6840-494a-a0ec-e21eaf463a4a service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:13.676952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-47ed3309-6840-494a-a0ec-e21eaf463a4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:13.689321 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed3309-6840-494a-a0ec-e21eaf463a4a service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:13.754263 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fdefb9-2926-4007-a754-73f21018af02 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:13.755769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-70fdefb9-2926-4007-a754-73f21018af02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:13.757908 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c9b47-6734-420d-b03e-b9591b8b5f0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:13.758572 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c9b47-6734-420d-b03e-b9591b8b5f0b service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:13.758860 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c9b47-6734-420d-b03e-b9591b8b5f0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:13.759187 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c9b47-6734-420d-b03e-b9591b8b5f0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:13.761121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ae4c9b47-6734-420d-b03e-b9591b8b5f0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:13.842344 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c9b47-6734-420d-b03e-b9591b8b5f0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:13.842991 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c9b47-6734-420d-b03e-b9591b8b5f0b service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:13.855239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1064/2147] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:13.875548 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fdefb9-2926-4007-a754-73f21018af02 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:13.883022 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe60648-c033-4c21-8e4c-423c1a6d192a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:13.883917 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe60648-c033-4c21-8e4c-423c1a6d192a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:13.884207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe60648-c033-4c21-8e4c-423c1a6d192a admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:13.884484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe60648-c033-4c21-8e4c-423c1a6d192a admin admin] PATH_INFO: `/v3/projects/161a5c1b39d14778bc0a27e8c3fa45a6` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:13.886094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe60648-c033-4c21-8e4c-423c1a6d192a admin admin] RBAC: Authorizing `identity:delete_project(project_id=161a5c1b39d14778bc0a27e8c3fa45a6)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:13.888080 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe60648-c033-4c21-8e4c-423c1a6d192a admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:13.932805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c9b47-6734-420d-b03e-b9591b8b5f0b service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:13.935736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ae4c9b47-6734-420d-b03e-b9591b8b5f0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:13.949270 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c9b47-6734-420d-b03e-b9591b8b5f0b service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:13.953179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b81ec5-da4c-40d0-9bce-b185ac2d5f5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:13.953957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b81ec5-da4c-40d0-9bce-b185ac2d5f5d service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:13.954120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b81ec5-da4c-40d0-9bce-b185ac2d5f5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:13.954377 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b81ec5-da4c-40d0-9bce-b185ac2d5f5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:13.957479 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-34b81ec5-da4c-40d0-9bce-b185ac2d5f5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:13.975276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-bbe60648-c033-4c21-8e4c-423c1a6d192a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '161a5c1b39d14778bc0a27e8c3fa45a6', 'request_id': 'req-bbe60648-c033-4c21-8e4c-423c1a6d192a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:13.977998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:13.985981 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-bbe60648-c033-4c21-8e4c-423c1a6d192a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '161a5c1b39d14778bc0a27e8c3fa45a6', 'request_id': 'req-bbe60648-c033-4c21-8e4c-423c1a6d192a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:14.000125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-bbe60648-c033-4c21-8e4c-423c1a6d192a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '161a5c1b39d14778bc0a27e8c3fa45a6', 'request_id': 'req-bbe60648-c033-4c21-8e4c-423c1a6d192a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:14.013264 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-bbe60648-c033-4c21-8e4c-423c1a6d192a admin admin] The token cache is being invalidate because project 161a5c1b39d14778bc0a27e8c3fa45a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:14.013460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-bbe60648-c033-4c21-8e4c-423c1a6d192a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:14.014070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-bbe60648-c033-4c21-8e4c-423c1a6d192a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:14.021100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1084/2148] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:13 2022] DELETE /identity/v3/projects/161a5c1b39d14778bc0a27e8c3fa45a6 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:36:14.023331 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d39fd-3452-4213-8596-8c9eef26b6b3 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:14.023729 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d39fd-3452-4213-8596-8c9eef26b6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:14.024028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d39fd-3452-4213-8596-8c9eef26b6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:14.048981 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-2b8d39fd-3452-4213-8596-8c9eef26b6b3 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:14.054516 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-2b8d39fd-3452-4213-8596-8c9eef26b6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:14.061938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b81ec5-da4c-40d0-9bce-b185ac2d5f5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:14.063696 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b81ec5-da4c-40d0-9bce-b185ac2d5f5d service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:14.083095 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1065/2149] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:36:14.147243 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1085/2150] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:14.153766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b57737-f946-4767-97ff-ef57df7ce8ef None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:14.154088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b57737-f946-4767-97ff-ef57df7ce8ef None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:14.154442 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b57737-f946-4767-97ff-ef57df7ce8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:14.177098 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f4b57737-f946-4767-97ff-ef57df7ce8ef None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:14.178461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f4b57737-f946-4767-97ff-ef57df7ce8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:14.180148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d39fd-3452-4213-8596-8c9eef26b6b3 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:14.181582 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-2b8d39fd-3452-4213-8596-8c9eef26b6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:14.209940 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d39fd-3452-4213-8596-8c9eef26b6b3 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:14.210164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1066/2151] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:14.211498 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b57737-f946-4767-97ff-ef57df7ce8ef None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:14.222742 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d170b9a-4c22-4368-967d-bcad6e8ec4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:14.223765 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d170b9a-4c22-4368-967d-bcad6e8ec4ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:14.224251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d170b9a-4c22-4368-967d-bcad6e8ec4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:14.224478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d170b9a-4c22-4368-967d-bcad6e8ec4ac admin admin] PATH_INFO: `/v3/users/d845e53dbc66437793dcd4e140187012` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:14.225884 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d170b9a-4c22-4368-967d-bcad6e8ec4ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=d845e53dbc66437793dcd4e140187012)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:14.229942 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d170b9a-4c22-4368-967d-bcad6e8ec4ac admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:14.231881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b57737-f946-4767-97ff-ef57df7ce8ef None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:14.250758 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff449ca-5d68-4501-9d3b-4d29d5322d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:14.252042 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff449ca-5d68-4501-9d3b-4d29d5322d6f admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:14.252334 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff449ca-5d68-4501-9d3b-4d29d5322d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:14.252613 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff449ca-5d68-4501-9d3b-4d29d5322d6f admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:14.253368 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff449ca-5d68-4501-9d3b-4d29d5322d6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:36:14.253682 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff449ca-5d68-4501-9d3b-4d29d5322d6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:14.255549 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff449ca-5d68-4501-9d3b-4d29d5322d6f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:14.280731 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1067/2152] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:36:14.288384 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff449ca-5d68-4501-9d3b-4d29d5322d6f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:14.305042 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff449ca-5d68-4501-9d3b-4d29d5322d6f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:14.313154 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f258263-16fe-4689-a1b0-d01f33f0d293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:14.314901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f258263-16fe-4689-a1b0-d01f33f0d293 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:14.315255 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f258263-16fe-4689-a1b0-d01f33f0d293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:14.315538 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f258263-16fe-4689-a1b0-d01f33f0d293 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:14.316252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f258263-16fe-4689-a1b0-d01f33f0d293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:14.318282 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f258263-16fe-4689-a1b0-d01f33f0d293 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:14.322309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0d170b9a-4c22-4368-967d-bcad6e8ec4ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd845e53dbc66437793dcd4e140187012'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:14.323238 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:14.333000 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0d170b9a-4c22-4368-967d-bcad6e8ec4ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd845e53dbc66437793dcd4e140187012'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:14.343350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0d170b9a-4c22-4368-967d-bcad6e8ec4ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd845e53dbc66437793dcd4e140187012'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:14.345218 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0d170b9a-4c22-4368-967d-bcad6e8ec4ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd845e53dbc66437793dcd4e140187012'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:14.345985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0d170b9a-4c22-4368-967d-bcad6e8ec4ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd845e53dbc66437793dcd4e140187012'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:14.359286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1068/2153] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:14 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:14.364725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f258263-16fe-4689-a1b0-d01f33f0d293 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:14.368461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6f258263-16fe-4689-a1b0-d01f33f0d293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:14.370135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1086/2154] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:14 2022] DELETE /identity/v3/users/d845e53dbc66437793dcd4e140187012 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:36:14.374388 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde60898-f82c-4dcc-9af6-a0163aced4f7 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:14.374708 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde60898-f82c-4dcc-9af6-a0163aced4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:14.375012 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde60898-f82c-4dcc-9af6-a0163aced4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:14.409926 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-bde60898-f82c-4dcc-9af6-a0163aced4f7 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:14.420170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-bde60898-f82c-4dcc-9af6-a0163aced4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:14.494754 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f258263-16fe-4689-a1b0-d01f33f0d293 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:14.524992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:14.552065 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1087/2155] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:14.559516 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde60898-f82c-4dcc-9af6-a0163aced4f7 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:14.561355 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-bde60898-f82c-4dcc-9af6-a0163aced4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:14.583467 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7497c7-e9cc-4d93-937f-86d446babbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:14.584644 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7497c7-e9cc-4d93-937f-86d446babbc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:14.584991 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7497c7-e9cc-4d93-937f-86d446babbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:14.585300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7497c7-e9cc-4d93-937f-86d446babbc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:14.586394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7497c7-e9cc-4d93-937f-86d446babbc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:14.588798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7497c7-e9cc-4d93-937f-86d446babbc9 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:14.617920 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1069/2156] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:14 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:14.623448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7497c7-e9cc-4d93-937f-86d446babbc9 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:14.638409 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7497c7-e9cc-4d93-937f-86d446babbc9 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:14.644120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde60898-f82c-4dcc-9af6-a0163aced4f7 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:14.646976 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdabd4-dde2-43a7-b807-168424845dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:14.647911 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdabd4-dde2-43a7-b807-168424845dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:14.648263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdabd4-dde2-43a7-b807-168424845dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:14.648610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdabd4-dde2-43a7-b807-168424845dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:14.648924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b7083-3e2e-4a2b-a781-625ff03001e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:14.649670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdabd4-dde2-43a7-b807-168424845dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:14.650112 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b7083-3e2e-4a2b-a781-625ff03001e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:14.650623 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b7083-3e2e-4a2b-a781-625ff03001e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:14.651015 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b7083-3e2e-4a2b-a781-625ff03001e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:14.651773 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdabd4-dde2-43a7-b807-168424845dc9 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:14.669129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1070/2157] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:14.676032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdabd4-dde2-43a7-b807-168424845dc9 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:14.676808 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b7083-3e2e-4a2b-a781-625ff03001e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:14.678392 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b7083-3e2e-4a2b-a781-625ff03001e1 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:14.689521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdabd4-dde2-43a7-b807-168424845dc9 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:14.691042 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1088/2158] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:36:14.697771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7656a-96d8-43cb-ab1e-e6fef8608f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:14.698721 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7656a-96d8-43cb-ab1e-e6fef8608f53 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:14.699235 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7656a-96d8-43cb-ab1e-e6fef8608f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:14.699668 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7656a-96d8-43cb-ab1e-e6fef8608f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:14.700922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7656a-96d8-43cb-ab1e-e6fef8608f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:14.703358 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7656a-96d8-43cb-ab1e-e6fef8608f53 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:14.720350 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1071/2159] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:36:14.723865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b7083-3e2e-4a2b-a781-625ff03001e1 service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:14.735482 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b7083-3e2e-4a2b-a781-625ff03001e1 service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:14.744543 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9346e-960c-4f51-9558-8ca6dd79b2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:14.745600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9346e-960c-4f51-9558-8ca6dd79b2a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:14.745982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9346e-960c-4f51-9558-8ca6dd79b2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:14.746388 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9346e-960c-4f51-9558-8ca6dd79b2a5 admin admin] PATH_INFO: `/v3/projects/b1983d5e5a914c9c8f2c9f01d54587dc/users/db3139faffef46698b1eed6d6668239e/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:14.769119 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9346e-960c-4f51-9558-8ca6dd79b2a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1983d5e5a914c9c8f2c9f01d54587dc, user_id=db3139faffef46698b1eed6d6668239e, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:14.772534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7656a-96d8-43cb-ab1e-e6fef8608f53 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:14.773832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9346e-960c-4f51-9558-8ca6dd79b2a5 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:14.774483 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-82c7656a-96d8-43cb-ab1e-e6fef8608f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:14.798985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1089/2160] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:14 2022] PUT /identity/v3/projects/b1983d5e5a914c9c8f2c9f01d54587dc/users/db3139faffef46698b1eed6d6668239e/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:14.807488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223b2a8-89f8-4bc1-9f72-0c06cdb404c0 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:14.807921 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223b2a8-89f8-4bc1-9f72-0c06cdb404c0 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:14.809458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223b2a8-89f8-4bc1-9f72-0c06cdb404c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:14.844029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-f223b2a8-89f8-4bc1-9f72-0c06cdb404c0 None None] MFA Rules not processed for user `db3139faffef46698b1eed6d6668239e`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:14.845559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f223b2a8-89f8-4bc1-9f72-0c06cdb404c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:14.853286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7656a-96d8-43cb-ab1e-e6fef8608f53 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:14.870612 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:14.931024 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dc2f6-bbab-4fa8-a12d-e61a0899bf38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:14.931692 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dc2f6-bbab-4fa8-a12d-e61a0899bf38 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:14.932437 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dc2f6-bbab-4fa8-a12d-e61a0899bf38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:14.932437 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dc2f6-bbab-4fa8-a12d-e61a0899bf38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:14.934270 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ea1dc2f6-bbab-4fa8-a12d-e61a0899bf38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:14.974845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1090/2161] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:36:14.985410 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223b2a8-89f8-4bc1-9f72-0c06cdb404c0 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:14.988283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f223b2a8-89f8-4bc1-9f72-0c06cdb404c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:15.005954 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223b2a8-89f8-4bc1-9f72-0c06cdb404c0 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:15.010105 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bbdc5-cf10-4ca5-88c0-9c426353aa2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:15.011499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bbdc5-cf10-4ca5-88c0-9c426353aa2a service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:15.011845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bbdc5-cf10-4ca5-88c0-9c426353aa2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:15.012294 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bbdc5-cf10-4ca5-88c0-9c426353aa2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:15.013651 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dc2f6-bbab-4fa8-a12d-e61a0899bf38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:15.015255 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dc2f6-bbab-4fa8-a12d-e61a0899bf38 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:15.023095 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6bbdc5-cf10-4ca5-88c0-9c426353aa2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:15.025264 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6bbdc5-cf10-4ca5-88c0-9c426353aa2a service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:15.031125 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1072/2162] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:36:15.048932 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1091/2163] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:15.138722 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dc2f6-bbab-4fa8-a12d-e61a0899bf38 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:15.140548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ea1dc2f6-bbab-4fa8-a12d-e61a0899bf38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:15.276399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dc2f6-bbab-4fa8-a12d-e61a0899bf38 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:15.284018 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6706ad-a0b5-4f1c-aaa1-23c2255c8863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:15.285586 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6706ad-a0b5-4f1c-aaa1-23c2255c8863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:15.286153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6706ad-a0b5-4f1c-aaa1-23c2255c8863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:15.286623 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6706ad-a0b5-4f1c-aaa1-23c2255c8863 admin admin] PATH_INFO: `/v3/projects/480c617c65b641c8bc0143ee0a8ec220` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:15.288125 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6706ad-a0b5-4f1c-aaa1-23c2255c8863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=480c617c65b641c8bc0143ee0a8ec220)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:15.290457 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6706ad-a0b5-4f1c-aaa1-23c2255c8863 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:15.371930 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7d6706ad-a0b5-4f1c-aaa1-23c2255c8863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '480c617c65b641c8bc0143ee0a8ec220', 'request_id': 'req-7d6706ad-a0b5-4f1c-aaa1-23c2255c8863'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:15.384057 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7d6706ad-a0b5-4f1c-aaa1-23c2255c8863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '480c617c65b641c8bc0143ee0a8ec220', 'request_id': 'req-7d6706ad-a0b5-4f1c-aaa1-23c2255c8863'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:15.392737 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7d6706ad-a0b5-4f1c-aaa1-23c2255c8863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '480c617c65b641c8bc0143ee0a8ec220', 'request_id': 'req-7d6706ad-a0b5-4f1c-aaa1-23c2255c8863'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:15.418708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:15.428647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7d6706ad-a0b5-4f1c-aaa1-23c2255c8863 admin admin] The token cache is being invalidate because project 480c617c65b641c8bc0143ee0a8ec220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:15.431157 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7d6706ad-a0b5-4f1c-aaa1-23c2255c8863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:15.432300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7d6706ad-a0b5-4f1c-aaa1-23c2255c8863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:15.437941 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1073/2164] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:15 2022] DELETE /identity/v3/projects/480c617c65b641c8bc0143ee0a8ec220 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:15.444059 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71c80d-f15b-4c6b-b70b-1f9659fd5d17 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:15.444410 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71c80d-f15b-4c6b-b70b-1f9659fd5d17 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:15.444702 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71c80d-f15b-4c6b-b70b-1f9659fd5d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:15.470223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-8a71c80d-f15b-4c6b-b70b-1f9659fd5d17 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:15.472769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-8a71c80d-f15b-4c6b-b70b-1f9659fd5d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:15.535610 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:15.574191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1092/2165] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:15.582257 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a4a90-9fa0-4745-b82a-13ad48647b9c None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:15.583420 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a4a90-9fa0-4745-b82a-13ad48647b9c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:15.583886 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a4a90-9fa0-4745-b82a-13ad48647b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:15.628218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-645a4a90-9fa0-4745-b82a-13ad48647b9c None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:15.632014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-645a4a90-9fa0-4745-b82a-13ad48647b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:15.651357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1074/2166] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:15.656031 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71c80d-f15b-4c6b-b70b-1f9659fd5d17 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:15.674145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71c80d-f15b-4c6b-b70b-1f9659fd5d17 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:15.689545 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ad02a-93f7-4769-841a-e162e34f6ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:15.690991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ad02a-93f7-4769-841a-e162e34f6ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:15.691399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ad02a-93f7-4769-841a-e162e34f6ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:15.691725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ad02a-93f7-4769-841a-e162e34f6ef8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:15.692489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ad02a-93f7-4769-841a-e162e34f6ef8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:36:15.692803 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ad02a-93f7-4769-841a-e162e34f6ef8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:15.694567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ad02a-93f7-4769-841a-e162e34f6ef8 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:15.738550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1093/2167] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:36:15.743702 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a4a90-9fa0-4745-b82a-13ad48647b9c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:15.758213 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a4a90-9fa0-4745-b82a-13ad48647b9c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:15.772928 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148a497-e82f-45d4-bb8c-8a054752a290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:15.773607 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148a497-e82f-45d4-bb8c-8a054752a290 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:15.773888 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148a497-e82f-45d4-bb8c-8a054752a290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:15.774163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148a497-e82f-45d4-bb8c-8a054752a290 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:15.774866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148a497-e82f-45d4-bb8c-8a054752a290 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:15.776807 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148a497-e82f-45d4-bb8c-8a054752a290 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:15.835118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1075/2168] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:15 2022] POST /identity/v3/projects => generated 399 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:15.836964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ad02a-93f7-4769-841a-e162e34f6ef8 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:15.849855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ad02a-93f7-4769-841a-e162e34f6ef8 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:15.934529 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb31ca-af17-4ff3-8fbc-95a35d21519b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:15.935263 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb31ca-af17-4ff3-8fbc-95a35d21519b admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:15.935560 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb31ca-af17-4ff3-8fbc-95a35d21519b admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:15.935837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb31ca-af17-4ff3-8fbc-95a35d21519b admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:15.936598 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb31ca-af17-4ff3-8fbc-95a35d21519b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:15.938719 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb31ca-af17-4ff3-8fbc-95a35d21519b admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:15.971388 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1094/2169] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:15 2022] POST /identity/v3/users => generated 300 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:15.972531 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148a497-e82f-45d4-bb8c-8a054752a290 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:15.983984 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148a497-e82f-45d4-bb8c-8a054752a290 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:15.990430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644aab8-55fe-499f-a179-f0577250ed59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:15.991279 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644aab8-55fe-499f-a179-f0577250ed59 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:15.991761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644aab8-55fe-499f-a179-f0577250ed59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:15.992212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644aab8-55fe-499f-a179-f0577250ed59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:15.993221 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644aab8-55fe-499f-a179-f0577250ed59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:15.995156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644aab8-55fe-499f-a179-f0577250ed59 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:16.010965 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1076/2170] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:36:16.012463 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb31ca-af17-4ff3-8fbc-95a35d21519b admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:16.023303 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb31ca-af17-4ff3-8fbc-95a35d21519b admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:16.027956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594acba8-0acf-4852-8547-b546f4bdcbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:16.028613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594acba8-0acf-4852-8547-b546f4bdcbc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:16.028896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594acba8-0acf-4852-8547-b546f4bdcbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:16.029173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594acba8-0acf-4852-8547-b546f4bdcbc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:16.030023 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594acba8-0acf-4852-8547-b546f4bdcbc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:16.031651 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594acba8-0acf-4852-8547-b546f4bdcbc2 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:16.042557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1095/2171] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:16.046484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644aab8-55fe-499f-a179-f0577250ed59 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:16.057071 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644aab8-55fe-499f-a179-f0577250ed59 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:16.061806 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b2146-6c7b-4168-b87e-57ca6ac08b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:16.062695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b2146-6c7b-4168-b87e-57ca6ac08b62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:16.063001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b2146-6c7b-4168-b87e-57ca6ac08b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:16.063282 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b2146-6c7b-4168-b87e-57ca6ac08b62 admin admin] PATH_INFO: `/v3/projects/fc295a123da6438ea9925f091301dcaf/users/1fac7295213842e4933df69eca61c9ee/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:16.079987 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b2146-6c7b-4168-b87e-57ca6ac08b62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc295a123da6438ea9925f091301dcaf, user_id=1fac7295213842e4933df69eca61c9ee, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:16.083904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b2146-6c7b-4168-b87e-57ca6ac08b62 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:16.097307 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1077/2172] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:16 2022] PUT /identity/v3/projects/fc295a123da6438ea9925f091301dcaf/users/1fac7295213842e4933df69eca61c9ee/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:36:16.108031 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac50c02-a492-4577-b5b0-9f5101c1dcc9 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:16.108428 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac50c02-a492-4577-b5b0-9f5101c1dcc9 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:16.108777 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac50c02-a492-4577-b5b0-9f5101c1dcc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:16.137498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-0ac50c02-a492-4577-b5b0-9f5101c1dcc9 None None] MFA Rules not processed for user `1fac7295213842e4933df69eca61c9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:16.138774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-0ac50c02-a492-4577-b5b0-9f5101c1dcc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:16.206621 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1096/2173] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:36:16.218177 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b2146-6c7b-4168-b87e-57ca6ac08b62 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:16.219950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4f7b2146-6c7b-4168-b87e-57ca6ac08b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:16.349867 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b2146-6c7b-4168-b87e-57ca6ac08b62 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:16.356117 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9f065-3dac-4b32-913f-7a4510e6385d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:16.357272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9f065-3dac-4b32-913f-7a4510e6385d service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:16.357790 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9f065-3dac-4b32-913f-7a4510e6385d service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:16.358249 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9f065-3dac-4b32-913f-7a4510e6385d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:16.374500 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9f065-3dac-4b32-913f-7a4510e6385d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:16.376786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9f065-3dac-4b32-913f-7a4510e6385d service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:16.391830 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1078/2174] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:17.750086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938dc87-1277-4c40-85a7-9f1e88aa93c5 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:17.760639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938dc87-1277-4c40-85a7-9f1e88aa93c5 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:17.760986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938dc87-1277-4c40-85a7-9f1e88aa93c5 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:17.763004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1097/2175] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 14 20:36:17 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:36:17.776205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f7715-bdb8-4702-b11c-f2fc1827be4a None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:17.777318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f7715-bdb8-4702-b11c-f2fc1827be4a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:17.777797 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f7715-bdb8-4702-b11c-f2fc1827be4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:17.806895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-cc0f7715-bdb8-4702-b11c-f2fc1827be4a None None] MFA Rules not processed for user `1afad68819504c2b864851c6989ae25a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:17.808449 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-cc0f7715-bdb8-4702-b11c-f2fc1827be4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:17.866751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938dc87-1277-4c40-85a7-9f1e88aa93c5 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:17.869357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-c938dc87-1277-4c40-85a7-9f1e88aa93c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:17.927557 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1079/2176] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 14 20:36:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:17.940196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f7715-bdb8-4702-b11c-f2fc1827be4a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:17.942222 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-cc0f7715-bdb8-4702-b11c-f2fc1827be4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:17.944142 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:17.960961 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f7715-bdb8-4702-b11c-f2fc1827be4a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:17.968812 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920a5aa-0c0c-4c99-a6e8-d3a185dac832 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afad68819504c2b864851c6989ae25a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:17.969928 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920a5aa-0c0c-4c99-a6e8-d3a185dac832 service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:17.970612 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920a5aa-0c0c-4c99-a6e8-d3a185dac832 service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:17.977394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920a5aa-0c0c-4c99-a6e8-d3a185dac832 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:17.980969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938dc87-1277-4c40-85a7-9f1e88aa93c5 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:17.989490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920a5aa-0c0c-4c99-a6e8-d3a185dac832 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:17.990071 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47dc942-1e03-4a11-9964-dac4e4f540cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:17.991018 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47dc942-1e03-4a11-9964-dac4e4f540cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:17.991397 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47dc942-1e03-4a11-9964-dac4e4f540cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:17.991526 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920a5aa-0c0c-4c99-a6e8-d3a185dac832 service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:17.991651 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47dc942-1e03-4a11-9964-dac4e4f540cb admin admin] PATH_INFO: `/v3/users/2fdc04cfa44f4a21a1ce9854fd43c068` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:17.993023 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47dc942-1e03-4a11-9964-dac4e4f540cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fdc04cfa44f4a21a1ce9854fd43c068)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:17.994970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47dc942-1e03-4a11-9964-dac4e4f540cb admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:18.003495 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1080/2177] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:36:18.053680 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-a47dc942-1e03-4a11-9964-dac4e4f540cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fdc04cfa44f4a21a1ce9854fd43c068'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.062633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-a47dc942-1e03-4a11-9964-dac4e4f540cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fdc04cfa44f4a21a1ce9854fd43c068'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.071260 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-a47dc942-1e03-4a11-9964-dac4e4f540cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fdc04cfa44f4a21a1ce9854fd43c068'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.071985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-a47dc942-1e03-4a11-9964-dac4e4f540cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fdc04cfa44f4a21a1ce9854fd43c068'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.072633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-a47dc942-1e03-4a11-9964-dac4e4f540cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fdc04cfa44f4a21a1ce9854fd43c068'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.090843 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1098/2178] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:17 2022] DELETE /identity/v3/users/2fdc04cfa44f4a21a1ce9854fd43c068 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:18.199149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920a5aa-0c0c-4c99-a6e8-d3a185dac832 service placement] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:18.201103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1920a5aa-0c0c-4c99-a6e8-d3a185dac832 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:18.232067 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920a5aa-0c0c-4c99-a6e8-d3a185dac832 service placement] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:18.259090 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:18.282912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47dc942-1e03-4a11-9964-dac4e4f540cb admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:18.284782 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-a47dc942-1e03-4a11-9964-dac4e4f540cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:18.328346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109c130-3de0-4c32-8831-707a9e402288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:18.329844 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109c130-3de0-4c32-8831-707a9e402288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:18.330134 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109c130-3de0-4c32-8831-707a9e402288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:18.330412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109c130-3de0-4c32-8831-707a9e402288 admin admin] PATH_INFO: `/v3/users/db3139faffef46698b1eed6d6668239e` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:18.331833 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109c130-3de0-4c32-8831-707a9e402288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db3139faffef46698b1eed6d6668239e)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:18.334041 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109c130-3de0-4c32-8831-707a9e402288 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:18.383926 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47dc942-1e03-4a11-9964-dac4e4f540cb admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:18.392429 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d85399-95b6-4255-824e-d029991c0796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:18.393380 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d85399-95b6-4255-824e-d029991c0796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:18.393622 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d85399-95b6-4255-824e-d029991c0796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:18.394015 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d85399-95b6-4255-824e-d029991c0796 admin admin] PATH_INFO: `/v3/projects/7ccdb0dd0a034f008329f489ed3f7d4d` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:18.396634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d85399-95b6-4255-824e-d029991c0796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ccdb0dd0a034f008329f489ed3f7d4d)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:18.398464 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d85399-95b6-4255-824e-d029991c0796 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:18.406653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1109c130-3de0-4c32-8831-707a9e402288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db3139faffef46698b1eed6d6668239e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.434289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1109c130-3de0-4c32-8831-707a9e402288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db3139faffef46698b1eed6d6668239e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.435045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1109c130-3de0-4c32-8831-707a9e402288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db3139faffef46698b1eed6d6668239e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.435765 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1109c130-3de0-4c32-8831-707a9e402288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db3139faffef46698b1eed6d6668239e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.453374 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1109c130-3de0-4c32-8831-707a9e402288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db3139faffef46698b1eed6d6668239e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.467783 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1081/2179] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:18 2022] DELETE /identity/v3/users/db3139faffef46698b1eed6d6668239e => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:18.475761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd96c3b-bb45-41e5-92f6-09d485ccdd6b None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:18.476164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd96c3b-bb45-41e5-92f6-09d485ccdd6b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:18.476525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd96c3b-bb45-41e5-92f6-09d485ccdd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:18.487988 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f5d85399-95b6-4255-824e-d029991c0796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ccdb0dd0a034f008329f489ed3f7d4d', 'request_id': 'req-f5d85399-95b6-4255-824e-d029991c0796'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.496184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f5d85399-95b6-4255-824e-d029991c0796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ccdb0dd0a034f008329f489ed3f7d4d', 'request_id': 'req-f5d85399-95b6-4255-824e-d029991c0796'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.500405 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-5dd96c3b-bb45-41e5-92f6-09d485ccdd6b None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:18.502618 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5dd96c3b-bb45-41e5-92f6-09d485ccdd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:18.504416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f5d85399-95b6-4255-824e-d029991c0796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ccdb0dd0a034f008329f489ed3f7d4d', 'request_id': 'req-f5d85399-95b6-4255-824e-d029991c0796'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.506924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:18.519574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f5d85399-95b6-4255-824e-d029991c0796 admin admin] The token cache is being invalidate because project 7ccdb0dd0a034f008329f489ed3f7d4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:18.519878 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f5d85399-95b6-4255-824e-d029991c0796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.520760 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f5d85399-95b6-4255-824e-d029991c0796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.522892 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1099/2180] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:18 2022] DELETE /identity/v3/projects/7ccdb0dd0a034f008329f489ed3f7d4d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:18.531868 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2bf47-87c1-4540-a6da-37391420ac27 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:18.531868 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2bf47-87c1-4540-a6da-37391420ac27 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:18.531868 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2bf47-87c1-4540-a6da-37391420ac27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:18.552701 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-84a2bf47-87c1-4540-a6da-37391420ac27 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:18.554495 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-84a2bf47-87c1-4540-a6da-37391420ac27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:18.571179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1082/2181] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:36:18.582750 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd96c3b-bb45-41e5-92f6-09d485ccdd6b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:18.584517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5dd96c3b-bb45-41e5-92f6-09d485ccdd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:18.597480 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd96c3b-bb45-41e5-92f6-09d485ccdd6b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:18.602143 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af6ecd-802a-4480-8b61-0a94db40cafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:18.603059 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af6ecd-802a-4480-8b61-0a94db40cafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:18.603346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af6ecd-802a-4480-8b61-0a94db40cafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:18.603647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af6ecd-802a-4480-8b61-0a94db40cafd admin admin] PATH_INFO: `/v3/users/188ff2b1d0b542a8923d5b097a7062b1` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:18.604860 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af6ecd-802a-4480-8b61-0a94db40cafd admin admin] RBAC: Authorizing `identity:delete_user(user_id=188ff2b1d0b542a8923d5b097a7062b1)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:18.606879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af6ecd-802a-4480-8b61-0a94db40cafd admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:18.618335 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1100/2182] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:18.619016 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2bf47-87c1-4540-a6da-37391420ac27 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:18.620278 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-84a2bf47-87c1-4540-a6da-37391420ac27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:18.657376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:18.670452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c5af6ecd-802a-4480-8b61-0a94db40cafd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '188ff2b1d0b542a8923d5b097a7062b1'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.679489 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c5af6ecd-802a-4480-8b61-0a94db40cafd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '188ff2b1d0b542a8923d5b097a7062b1'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.680365 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c5af6ecd-802a-4480-8b61-0a94db40cafd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '188ff2b1d0b542a8923d5b097a7062b1'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.681032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c5af6ecd-802a-4480-8b61-0a94db40cafd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '188ff2b1d0b542a8923d5b097a7062b1'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.687321 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2bf47-87c1-4540-a6da-37391420ac27 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:18.691019 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729f4f8-949d-4168-963c-20b4db0cd60f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:18.691669 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729f4f8-949d-4168-963c-20b4db0cd60f service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:18.691951 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729f4f8-949d-4168-963c-20b4db0cd60f service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:18.692228 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729f4f8-949d-4168-963c-20b4db0cd60f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:18.694031 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-2729f4f8-949d-4168-963c-20b4db0cd60f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:18.694393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c5af6ecd-802a-4480-8b61-0a94db40cafd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '188ff2b1d0b542a8923d5b097a7062b1'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:18.706584 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729f4f8-949d-4168-963c-20b4db0cd60f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:18.708066 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729f4f8-949d-4168-963c-20b4db0cd60f service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:18.709881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1083/2183] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:18 2022] DELETE /identity/v3/users/188ff2b1d0b542a8923d5b097a7062b1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:36:18.711287 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ce727-dcae-427a-9dc3-c94a6b695ca1 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:18.711576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ce727-dcae-427a-9dc3-c94a6b695ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:18.711848 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ce727-dcae-427a-9dc3-c94a6b695ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:18.718450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1101/2184] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:18.719740 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ff4bb-08b2-46f2-a768-3f9dbc76868d None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:18.720562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ff4bb-08b2-46f2-a768-3f9dbc76868d None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:18.720978 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ff4bb-08b2-46f2-a768-3f9dbc76868d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:18.730748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ce3ce727-dcae-427a-9dc3-c94a6b695ca1 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:18.732051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ce3ce727-dcae-427a-9dc3-c94a6b695ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:18.739260 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-475ff4bb-08b2-46f2-a768-3f9dbc76868d None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:18.740530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-475ff4bb-08b2-46f2-a768-3f9dbc76868d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:18.810734 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1084/2185] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:18.812288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ce727-dcae-427a-9dc3-c94a6b695ca1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:18.820754 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1102/2186] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:18.825289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ce727-dcae-427a-9dc3-c94a6b695ca1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:18.828983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ff4bb-08b2-46f2-a768-3f9dbc76868d None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:18.830687 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2d409-1f84-4a38-a5e2-ebb0702dce67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:18.831465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-475ff4bb-08b2-46f2-a768-3f9dbc76868d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:18.831751 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2d409-1f84-4a38-a5e2-ebb0702dce67 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:18.832055 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2d409-1f84-4a38-a5e2-ebb0702dce67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:18.832470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2d409-1f84-4a38-a5e2-ebb0702dce67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:18.833370 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2d409-1f84-4a38-a5e2-ebb0702dce67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:36:18.833850 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2d409-1f84-4a38-a5e2-ebb0702dce67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:18.835706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2d409-1f84-4a38-a5e2-ebb0702dce67 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:18.849150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1085/2187] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:36:18.854205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2d409-1f84-4a38-a5e2-ebb0702dce67 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:18.865364 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2d409-1f84-4a38-a5e2-ebb0702dce67 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:18.870842 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a40e1e-7fc7-4531-850d-1b8fb20266db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:18.871617 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a40e1e-7fc7-4531-850d-1b8fb20266db admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:18.872051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a40e1e-7fc7-4531-850d-1b8fb20266db admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:18.872320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a40e1e-7fc7-4531-850d-1b8fb20266db admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:18.873081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a40e1e-7fc7-4531-850d-1b8fb20266db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:18.875069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a40e1e-7fc7-4531-850d-1b8fb20266db admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:18.878415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:18.899073 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ff4bb-08b2-46f2-a768-3f9dbc76868d None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:18.903374 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e51d1-29c5-451e-9ee0-1fff7d27ee11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:18.904639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e51d1-29c5-451e-9ee0-1fff7d27ee11 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:18.904639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e51d1-29c5-451e-9ee0-1fff7d27ee11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:18.904786 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e51d1-29c5-451e-9ee0-1fff7d27ee11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:18.923085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e51d1-29c5-451e-9ee0-1fff7d27ee11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:18.924448 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e51d1-29c5-451e-9ee0-1fff7d27ee11 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:18.925192 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1086/2188] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:18 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:18.926053 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a40e1e-7fc7-4531-850d-1b8fb20266db admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:18.943571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1103/2189] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:19.003595 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a40e1e-7fc7-4531-850d-1b8fb20266db admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:19.008989 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320e9e3-ae93-4e7d-bec7-af8c9bd69794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:19.009728 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320e9e3-ae93-4e7d-bec7-af8c9bd69794 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:19.010014 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320e9e3-ae93-4e7d-bec7-af8c9bd69794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:19.010294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320e9e3-ae93-4e7d-bec7-af8c9bd69794 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:19.011209 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320e9e3-ae93-4e7d-bec7-af8c9bd69794 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:19.013240 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320e9e3-ae93-4e7d-bec7-af8c9bd69794 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:19.038727 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1087/2190] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:18 2022] POST /identity/v3/users => generated 291 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:19.041181 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e51d1-29c5-451e-9ee0-1fff7d27ee11 service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:19.051974 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e51d1-29c5-451e-9ee0-1fff7d27ee11 service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:19.057698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e4d17d-17aa-47af-81a0-5990e2ef4dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:19.058372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e4d17d-17aa-47af-81a0-5990e2ef4dee admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:19.059155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e4d17d-17aa-47af-81a0-5990e2ef4dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:19.059440 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e4d17d-17aa-47af-81a0-5990e2ef4dee admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:19.060183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e4d17d-17aa-47af-81a0-5990e2ef4dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:19.061785 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e4d17d-17aa-47af-81a0-5990e2ef4dee admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:19.074963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1104/2191] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:19.087918 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320e9e3-ae93-4e7d-bec7-af8c9bd69794 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:19.101988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320e9e3-ae93-4e7d-bec7-af8c9bd69794 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:19.110480 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9494e-65c6-4f96-ba96-90be19bc37a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:19.111301 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9494e-65c6-4f96-ba96-90be19bc37a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:19.111589 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9494e-65c6-4f96-ba96-90be19bc37a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:19.112025 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9494e-65c6-4f96-ba96-90be19bc37a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:19.112633 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9494e-65c6-4f96-ba96-90be19bc37a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:19.116000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9494e-65c6-4f96-ba96-90be19bc37a2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:19.119246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:19.130983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1088/2192] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:19.136110 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e4d17d-17aa-47af-81a0-5990e2ef4dee admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:19.148538 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e4d17d-17aa-47af-81a0-5990e2ef4dee admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:19.153844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc8a6f-2d1f-4aee-8d36-4c05c6e149a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:19.154877 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc8a6f-2d1f-4aee-8d36-4c05c6e149a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:19.155363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc8a6f-2d1f-4aee-8d36-4c05c6e149a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:19.155881 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc8a6f-2d1f-4aee-8d36-4c05c6e149a4 admin admin] PATH_INFO: `/v3/projects/4c5bfa264f5746fa98c845ae8bd953eb/users/af83c0aaa726410fbb1e4e7ce61d0c25/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:19.156718 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9494e-65c6-4f96-ba96-90be19bc37a2 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:19.159154 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-96b9494e-65c6-4f96-ba96-90be19bc37a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:19.172269 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9494e-65c6-4f96-ba96-90be19bc37a2 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:19.178784 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64ad65-e2b9-45ab-b7e4-df7d0bdb48d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:19.178924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc8a6f-2d1f-4aee-8d36-4c05c6e149a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5bfa264f5746fa98c845ae8bd953eb, user_id=af83c0aaa726410fbb1e4e7ce61d0c25, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:19.179720 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64ad65-e2b9-45ab-b7e4-df7d0bdb48d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:19.179983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64ad65-e2b9-45ab-b7e4-df7d0bdb48d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:19.180262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64ad65-e2b9-45ab-b7e4-df7d0bdb48d4 admin admin] PATH_INFO: `/v3/projects/b1983d5e5a914c9c8f2c9f01d54587dc` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:19.181540 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64ad65-e2b9-45ab-b7e4-df7d0bdb48d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1983d5e5a914c9c8f2c9f01d54587dc)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:19.183442 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64ad65-e2b9-45ab-b7e4-df7d0bdb48d4 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:19.183751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc8a6f-2d1f-4aee-8d36-4c05c6e149a4 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:19.225012 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1105/2193] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:19 2022] PUT /identity/v3/projects/4c5bfa264f5746fa98c845ae8bd953eb/users/af83c0aaa726410fbb1e4e7ce61d0c25/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:19.249265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3551a-69ae-4447-ad2a-e62e4bb6e5a0 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:19.249679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3551a-69ae-4447-ad2a-e62e4bb6e5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:19.249876 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3551a-69ae-4447-ad2a-e62e4bb6e5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:19.259474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-af64ad65-e2b9-45ab-b7e4-df7d0bdb48d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1983d5e5a914c9c8f2c9f01d54587dc', 'request_id': 'req-af64ad65-e2b9-45ab-b7e4-df7d0bdb48d4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.268865 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-af64ad65-e2b9-45ab-b7e4-df7d0bdb48d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1983d5e5a914c9c8f2c9f01d54587dc', 'request_id': 'req-af64ad65-e2b9-45ab-b7e4-df7d0bdb48d4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.276098 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-af64ad65-e2b9-45ab-b7e4-df7d0bdb48d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1983d5e5a914c9c8f2c9f01d54587dc', 'request_id': 'req-af64ad65-e2b9-45ab-b7e4-df7d0bdb48d4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.287329 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-1ea3551a-69ae-4447-ad2a-e62e4bb6e5a0 None None] MFA Rules not processed for user `af83c0aaa726410fbb1e4e7ce61d0c25`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:19.288761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-1ea3551a-69ae-4447-ad2a-e62e4bb6e5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:19.293921 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-af64ad65-e2b9-45ab-b7e4-df7d0bdb48d4 admin admin] The token cache is being invalidate because project b1983d5e5a914c9c8f2c9f01d54587dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:19.295104 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-af64ad65-e2b9-45ab-b7e4-df7d0bdb48d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.296004 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-af64ad65-e2b9-45ab-b7e4-df7d0bdb48d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.299005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1089/2194] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:19 2022] DELETE /identity/v3/projects/b1983d5e5a914c9c8f2c9f01d54587dc => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:36:19.299949 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64ad65-e2b9-45ab-b7e4-df7d0bdb48d4 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:19.301481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-af64ad65-e2b9-45ab-b7e4-df7d0bdb48d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:19.383957 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1106/2195] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:19.384771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64ad65-e2b9-45ab-b7e4-df7d0bdb48d4 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:19.385644 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebe77e-895e-4f2b-856d-62d4f7e4c6bc None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:19.385999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebe77e-895e-4f2b-856d-62d4f7e4c6bc None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:19.386276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebe77e-895e-4f2b-856d-62d4f7e4c6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:19.391001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8204bda-6cc1-4056-a3aa-ca45ff7d38a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:19.391931 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8204bda-6cc1-4056-a3aa-ca45ff7d38a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:19.392210 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8204bda-6cc1-4056-a3aa-ca45ff7d38a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:19.393068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8204bda-6cc1-4056-a3aa-ca45ff7d38a2 admin admin] PATH_INFO: `/v3/users/bb72c2809fe94d008e2acc5fc24c2316` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:19.393896 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8204bda-6cc1-4056-a3aa-ca45ff7d38a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb72c2809fe94d008e2acc5fc24c2316)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:19.396015 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8204bda-6cc1-4056-a3aa-ca45ff7d38a2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:19.414467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-e8ebe77e-895e-4f2b-856d-62d4f7e4c6bc None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:19.415775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e8ebe77e-895e-4f2b-856d-62d4f7e4c6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:19.427758 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1107/2196] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:19.428521 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebe77e-895e-4f2b-856d-62d4f7e4c6bc None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:19.429801 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e8ebe77e-895e-4f2b-856d-62d4f7e4c6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:19.443295 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebe77e-895e-4f2b-856d-62d4f7e4c6bc None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:19.449211 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d33d11-dbfa-4a57-9da8-83356393e3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:19.450146 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d33d11-dbfa-4a57-9da8-83356393e3d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:19.450438 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d33d11-dbfa-4a57-9da8-83356393e3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:19.450883 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d33d11-dbfa-4a57-9da8-83356393e3d1 admin admin] PATH_INFO: `/v3/projects/22784ca7f8ba4bebb836c35bd673f0fc` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:19.452456 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d33d11-dbfa-4a57-9da8-83356393e3d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22784ca7f8ba4bebb836c35bd673f0fc)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:19.453025 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c8204bda-6cc1-4056-a3aa-ca45ff7d38a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb72c2809fe94d008e2acc5fc24c2316'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.455126 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d33d11-dbfa-4a57-9da8-83356393e3d1 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:19.461848 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c8204bda-6cc1-4056-a3aa-ca45ff7d38a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb72c2809fe94d008e2acc5fc24c2316'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.462969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c8204bda-6cc1-4056-a3aa-ca45ff7d38a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb72c2809fe94d008e2acc5fc24c2316'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.463704 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c8204bda-6cc1-4056-a3aa-ca45ff7d38a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb72c2809fe94d008e2acc5fc24c2316'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.481632 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c8204bda-6cc1-4056-a3aa-ca45ff7d38a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb72c2809fe94d008e2acc5fc24c2316'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.490509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:19.496712 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1090/2197] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:19 2022] DELETE /identity/v3/users/bb72c2809fe94d008e2acc5fc24c2316 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:19.497725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8204bda-6cc1-4056-a3aa-ca45ff7d38a2 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:19.501338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c8204bda-6cc1-4056-a3aa-ca45ff7d38a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:19.508863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e7d33d11-dbfa-4a57-9da8-83356393e3d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22784ca7f8ba4bebb836c35bd673f0fc', 'request_id': 'req-e7d33d11-dbfa-4a57-9da8-83356393e3d1'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.516599 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e7d33d11-dbfa-4a57-9da8-83356393e3d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22784ca7f8ba4bebb836c35bd673f0fc', 'request_id': 'req-e7d33d11-dbfa-4a57-9da8-83356393e3d1'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.525478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e7d33d11-dbfa-4a57-9da8-83356393e3d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22784ca7f8ba4bebb836c35bd673f0fc', 'request_id': 'req-e7d33d11-dbfa-4a57-9da8-83356393e3d1'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.537298 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e7d33d11-dbfa-4a57-9da8-83356393e3d1 admin admin] The token cache is being invalidate because project 22784ca7f8ba4bebb836c35bd673f0fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:19.537598 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e7d33d11-dbfa-4a57-9da8-83356393e3d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.538339 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e7d33d11-dbfa-4a57-9da8-83356393e3d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.540231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1108/2198] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:19 2022] DELETE /identity/v3/projects/22784ca7f8ba4bebb836c35bd673f0fc => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:19.541249 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10984156-82cf-4fd1-889e-cd1a2c3f55fe None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:19.541527 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10984156-82cf-4fd1-889e-cd1a2c3f55fe None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:19.541807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10984156-82cf-4fd1-889e-cd1a2c3f55fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:19.561203 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-10984156-82cf-4fd1-889e-cd1a2c3f55fe None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:19.562406 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-10984156-82cf-4fd1-889e-cd1a2c3f55fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:19.568213 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8204bda-6cc1-4056-a3aa-ca45ff7d38a2 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:19.570401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1109/2199] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:19.571658 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8733834-45de-4527-b2cd-411db676d720 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:19.571948 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8733834-45de-4527-b2cd-411db676d720 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:19.572226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8733834-45de-4527-b2cd-411db676d720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:19.573088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39149e9a-30c8-4614-88e4-59e434b6df34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:19.573820 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39149e9a-30c8-4614-88e4-59e434b6df34 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:19.574024 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39149e9a-30c8-4614-88e4-59e434b6df34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:19.574301 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39149e9a-30c8-4614-88e4-59e434b6df34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:19.575076 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39149e9a-30c8-4614-88e4-59e434b6df34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:19.576987 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39149e9a-30c8-4614-88e4-59e434b6df34 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:19.577773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:19.592646 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-b8733834-45de-4527-b2cd-411db676d720 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:19.593834 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b8733834-45de-4527-b2cd-411db676d720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:19.601942 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1110/2200] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:19.602713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8733834-45de-4527-b2cd-411db676d720 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:19.606529 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1091/2201] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:19 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:19.606778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b8733834-45de-4527-b2cd-411db676d720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:19.607446 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39149e9a-30c8-4614-88e4-59e434b6df34 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:19.619533 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8733834-45de-4527-b2cd-411db676d720 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:19.674845 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39149e9a-30c8-4614-88e4-59e434b6df34 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:19.681047 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fb368-5ae4-4995-9122-e44681e73854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:19.681487 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ff1b2-1564-4007-ad29-ccd742bd9efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:19.681653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fb368-5ae4-4995-9122-e44681e73854 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:19.681937 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fb368-5ae4-4995-9122-e44681e73854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:19.682222 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fb368-5ae4-4995-9122-e44681e73854 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:19.682371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ff1b2-1564-4007-ad29-ccd742bd9efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:19.682658 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ff1b2-1564-4007-ad29-ccd742bd9efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:19.683019 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ff1b2-1564-4007-ad29-ccd742bd9efb admin admin] PATH_INFO: `/v3/users/b30f6783fb644c8aa395ca3c5f43f0e4` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:19.683120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2fb368-5ae4-4995-9122-e44681e73854 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:36:19.683420 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2fb368-5ae4-4995-9122-e44681e73854 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:19.684333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ff1b2-1564-4007-ad29-ccd742bd9efb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b30f6783fb644c8aa395ca3c5f43f0e4)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:19.685025 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2fb368-5ae4-4995-9122-e44681e73854 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:19.686249 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ff1b2-1564-4007-ad29-ccd742bd9efb admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:19.697480 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1092/2202] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:36:19.698323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fb368-5ae4-4995-9122-e44681e73854 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:19.702303 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6a2fb368-5ae4-4995-9122-e44681e73854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:19.716360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fb368-5ae4-4995-9122-e44681e73854 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:19.722583 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95428875-ca43-4539-8130-daa2fe445e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:19.723293 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95428875-ca43-4539-8130-daa2fe445e48 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:19.723593 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95428875-ca43-4539-8130-daa2fe445e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:19.723868 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95428875-ca43-4539-8130-daa2fe445e48 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:19.724562 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95428875-ca43-4539-8130-daa2fe445e48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:19.726480 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95428875-ca43-4539-8130-daa2fe445e48 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:19.738780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e04ff1b2-1564-4007-ad29-ccd742bd9efb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b30f6783fb644c8aa395ca3c5f43f0e4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.746470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e04ff1b2-1564-4007-ad29-ccd742bd9efb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b30f6783fb644c8aa395ca3c5f43f0e4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.747271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1093/2203] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:19 2022] POST /identity/v3/users => generated 297 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:19.748092 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95428875-ca43-4539-8130-daa2fe445e48 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:19.749706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-95428875-ca43-4539-8130-daa2fe445e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:19.757373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e04ff1b2-1564-4007-ad29-ccd742bd9efb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b30f6783fb644c8aa395ca3c5f43f0e4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.758070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e04ff1b2-1564-4007-ad29-ccd742bd9efb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b30f6783fb644c8aa395ca3c5f43f0e4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.758699 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-e04ff1b2-1564-4007-ad29-ccd742bd9efb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b30f6783fb644c8aa395ca3c5f43f0e4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:19.777226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1111/2204] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:19 2022] DELETE /identity/v3/users/b30f6783fb644c8aa395ca3c5f43f0e4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:19.778018 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ff1b2-1564-4007-ad29-ccd742bd9efb admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:19.779414 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e04ff1b2-1564-4007-ad29-ccd742bd9efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:19.808873 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95428875-ca43-4539-8130-daa2fe445e48 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:19.821697 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:19.850731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ff1b2-1564-4007-ad29-ccd742bd9efb admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:19.858209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285edbd-cdfb-40c4-bea2-39ac6491b6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:19.859134 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285edbd-cdfb-40c4-bea2-39ac6491b6ea admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:19.859493 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285edbd-cdfb-40c4-bea2-39ac6491b6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:19.859849 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285edbd-cdfb-40c4-bea2-39ac6491b6ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:19.860600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285edbd-cdfb-40c4-bea2-39ac6491b6ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:19.862563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285edbd-cdfb-40c4-bea2-39ac6491b6ea admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:19.870510 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309edea-33d3-4d3b-b2aa-0733659edcb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:19.871397 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309edea-33d3-4d3b-b2aa-0733659edcb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:19.871515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309edea-33d3-4d3b-b2aa-0733659edcb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:19.871732 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309edea-33d3-4d3b-b2aa-0733659edcb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:19.873604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9309edea-33d3-4d3b-b2aa-0733659edcb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:19.883394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:19.888031 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309edea-33d3-4d3b-b2aa-0733659edcb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:19.889968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309edea-33d3-4d3b-b2aa-0733659edcb8 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:19.891347 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1112/2205] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:19 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:19.897981 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285edbd-cdfb-40c4-bea2-39ac6491b6ea admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:19.901633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3285edbd-cdfb-40c4-bea2-39ac6491b6ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:19.902684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1094/2206] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:19.903389 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309edea-33d3-4d3b-b2aa-0733659edcb8 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:19.914744 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309edea-33d3-4d3b-b2aa-0733659edcb8 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:19.986813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285edbd-cdfb-40c4-bea2-39ac6491b6ea admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:19.994983 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aba44a-f233-4ce1-8e71-196c1b842d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:19.996138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aba44a-f233-4ce1-8e71-196c1b842d75 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:19.996435 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aba44a-f233-4ce1-8e71-196c1b842d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:19.996719 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aba44a-f233-4ce1-8e71-196c1b842d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:19.997802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aba44a-f233-4ce1-8e71-196c1b842d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:19.999559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aba44a-f233-4ce1-8e71-196c1b842d75 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:20.007245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b2eef-4eb8-4c43-bf84-fec1125e7184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:20.008039 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b2eef-4eb8-4c43-bf84-fec1125e7184 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:20.009334 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b2eef-4eb8-4c43-bf84-fec1125e7184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:20.009650 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b2eef-4eb8-4c43-bf84-fec1125e7184 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:20.010457 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b2eef-4eb8-4c43-bf84-fec1125e7184 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:20.012603 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b2eef-4eb8-4c43-bf84-fec1125e7184 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:20.015089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1113/2207] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:20.019887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aba44a-f233-4ce1-8e71-196c1b842d75 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:20.036136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aba44a-f233-4ce1-8e71-196c1b842d75 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:20.041015 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1095/2208] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:19 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:20.043953 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cc73c-1296-4bba-b01b-85d8b9c12da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:20.044999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cc73c-1296-4bba-b01b-85d8b9c12da4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:20.045150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b2eef-4eb8-4c43-bf84-fec1125e7184 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:20.045871 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cc73c-1296-4bba-b01b-85d8b9c12da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:20.046218 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cc73c-1296-4bba-b01b-85d8b9c12da4 admin admin] PATH_INFO: `/v3/projects/690e437c529c4ec59f8d1087fd970b47/users/2ec1cd2998844a7d85840ac284083d3b/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:20.056308 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b2eef-4eb8-4c43-bf84-fec1125e7184 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:20.061694 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747933b-9a4b-43e6-bf2f-83d44219758a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:20.062372 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747933b-9a4b-43e6-bf2f-83d44219758a admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:20.062659 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747933b-9a4b-43e6-bf2f-83d44219758a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:20.062964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747933b-9a4b-43e6-bf2f-83d44219758a admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:20.064494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747933b-9a4b-43e6-bf2f-83d44219758a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:20.065938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747933b-9a4b-43e6-bf2f-83d44219758a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:20.069071 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cc73c-1296-4bba-b01b-85d8b9c12da4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=690e437c529c4ec59f8d1087fd970b47, user_id=2ec1cd2998844a7d85840ac284083d3b, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:20.074371 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cc73c-1296-4bba-b01b-85d8b9c12da4 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:20.079113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1096/2209] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:20.081875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747933b-9a4b-43e6-bf2f-83d44219758a admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:20.107649 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747933b-9a4b-43e6-bf2f-83d44219758a admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:20.117571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1114/2210] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:20 2022] PUT /identity/v3/projects/690e437c529c4ec59f8d1087fd970b47/users/2ec1cd2998844a7d85840ac284083d3b/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:36:20.125528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cc73c-1296-4bba-b01b-85d8b9c12da4 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:20.222727 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31d406-1bef-4b12-bcac-90352bdf1dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:20.225534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31d406-1bef-4b12-bcac-90352bdf1dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:20.226507 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31d406-1bef-4b12-bcac-90352bdf1dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:20.226798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31d406-1bef-4b12-bcac-90352bdf1dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:20.231827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd31d406-1bef-4b12-bcac-90352bdf1dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:20.232557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cc73c-1296-4bba-b01b-85d8b9c12da4 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:20.233507 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd31d406-1bef-4b12-bcac-90352bdf1dc7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:20.239129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760b6cb-83b2-47df-852a-b1325cb0f12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:20.240790 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760b6cb-83b2-47df-852a-b1325cb0f12e admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:20.241100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760b6cb-83b2-47df-852a-b1325cb0f12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:20.241378 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760b6cb-83b2-47df-852a-b1325cb0f12e admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:20.242214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760b6cb-83b2-47df-852a-b1325cb0f12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:20.243987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760b6cb-83b2-47df-852a-b1325cb0f12e admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:20.254326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1097/2211] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:20.255233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31d406-1bef-4b12-bcac-90352bdf1dc7 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:20.258360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fd31d406-1bef-4b12-bcac-90352bdf1dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:20.267648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1115/2212] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:20.269167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760b6cb-83b2-47df-852a-b1325cb0f12e admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:20.271379 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-1760b6cb-83b2-47df-852a-b1325cb0f12e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:20.284975 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31d406-1bef-4b12-bcac-90352bdf1dc7 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:20.290335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee0610-72f5-4649-8bf2-82d5db8c7bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:20.291322 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee0610-72f5-4649-8bf2-82d5db8c7bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:20.291678 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee0610-72f5-4649-8bf2-82d5db8c7bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:20.292038 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee0610-72f5-4649-8bf2-82d5db8c7bda admin admin] PATH_INFO: `/v3/projects/14699e92ffcd4de7b4ed6b48fc6e29f0` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:20.293464 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee0610-72f5-4649-8bf2-82d5db8c7bda admin admin] RBAC: Authorizing `identity:delete_project(project_id=14699e92ffcd4de7b4ed6b48fc6e29f0)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:20.294992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760b6cb-83b2-47df-852a-b1325cb0f12e admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:20.295570 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee0610-72f5-4649-8bf2-82d5db8c7bda admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:20.300532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f72a5-c9ad-47cf-a234-48eefd18079c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:20.301480 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f72a5-c9ad-47cf-a234-48eefd18079c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:20.301831 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f72a5-c9ad-47cf-a234-48eefd18079c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:20.302176 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f72a5-c9ad-47cf-a234-48eefd18079c admin admin] PATH_INFO: `/v3/projects/b13d332f75044502bc92a00ca050a7f9` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:20.303469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f72a5-c9ad-47cf-a234-48eefd18079c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b13d332f75044502bc92a00ca050a7f9)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:20.305577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f72a5-c9ad-47cf-a234-48eefd18079c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:20.333726 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:20.352754 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-27ee0610-72f5-4649-8bf2-82d5db8c7bda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14699e92ffcd4de7b4ed6b48fc6e29f0', 'request_id': 'req-27ee0610-72f5-4649-8bf2-82d5db8c7bda'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.359315 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-27ee0610-72f5-4649-8bf2-82d5db8c7bda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14699e92ffcd4de7b4ed6b48fc6e29f0', 'request_id': 'req-27ee0610-72f5-4649-8bf2-82d5db8c7bda'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.361458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cb3f72a5-c9ad-47cf-a234-48eefd18079c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b13d332f75044502bc92a00ca050a7f9', 'request_id': 'req-cb3f72a5-c9ad-47cf-a234-48eefd18079c'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.365470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-27ee0610-72f5-4649-8bf2-82d5db8c7bda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14699e92ffcd4de7b4ed6b48fc6e29f0', 'request_id': 'req-27ee0610-72f5-4649-8bf2-82d5db8c7bda'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.369020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cb3f72a5-c9ad-47cf-a234-48eefd18079c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b13d332f75044502bc92a00ca050a7f9', 'request_id': 'req-cb3f72a5-c9ad-47cf-a234-48eefd18079c'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.376001 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cb3f72a5-c9ad-47cf-a234-48eefd18079c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b13d332f75044502bc92a00ca050a7f9', 'request_id': 'req-cb3f72a5-c9ad-47cf-a234-48eefd18079c'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.378742 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:20.379076 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-27ee0610-72f5-4649-8bf2-82d5db8c7bda admin admin] The token cache is being invalidate because project 14699e92ffcd4de7b4ed6b48fc6e29f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:20.379372 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-27ee0610-72f5-4649-8bf2-82d5db8c7bda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.380128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-27ee0610-72f5-4649-8bf2-82d5db8c7bda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.381833 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1098/2213] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:20 2022] DELETE /identity/v3/projects/14699e92ffcd4de7b4ed6b48fc6e29f0 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:20.382553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee0610-72f5-4649-8bf2-82d5db8c7bda admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:20.384065 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-27ee0610-72f5-4649-8bf2-82d5db8c7bda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:20.391117 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cb3f72a5-c9ad-47cf-a234-48eefd18079c admin admin] The token cache is being invalidate because project b13d332f75044502bc92a00ca050a7f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:20.391407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cb3f72a5-c9ad-47cf-a234-48eefd18079c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.392202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cb3f72a5-c9ad-47cf-a234-48eefd18079c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.393781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1116/2214] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:20 2022] DELETE /identity/v3/projects/b13d332f75044502bc92a00ca050a7f9 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:20.394597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f72a5-c9ad-47cf-a234-48eefd18079c admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:20.398182 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-cb3f72a5-c9ad-47cf-a234-48eefd18079c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:20.451045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee0610-72f5-4649-8bf2-82d5db8c7bda admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:20.463046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f72a5-c9ad-47cf-a234-48eefd18079c admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:20.467778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f05631-8ee6-47ff-a6ae-ee99c85b30fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:20.468698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f05631-8ee6-47ff-a6ae-ee99c85b30fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:20.469137 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f05631-8ee6-47ff-a6ae-ee99c85b30fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:20.469420 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f05631-8ee6-47ff-a6ae-ee99c85b30fc admin admin] PATH_INFO: `/v3/projects/690e437c529c4ec59f8d1087fd970b47/users/2ec1cd2998844a7d85840ac284083d3b/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:20.472263 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f05631-8ee6-47ff-a6ae-ee99c85b30fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=690e437c529c4ec59f8d1087fd970b47, user_id=2ec1cd2998844a7d85840ac284083d3b, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:20.474928 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f05631-8ee6-47ff-a6ae-ee99c85b30fc admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:20.496265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1117/2215] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:20 2022] PUT /identity/v3/projects/690e437c529c4ec59f8d1087fd970b47/users/2ec1cd2998844a7d85840ac284083d3b/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:20.496265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f05631-8ee6-47ff-a6ae-ee99c85b30fc admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:20.497444 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-18f05631-8ee6-47ff-a6ae-ee99c85b30fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:20.512919 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f8cfd-5fe4-4277-89f9-436f4f6f84d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:20.513821 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f8cfd-5fe4-4277-89f9-436f4f6f84d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:20.514105 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f8cfd-5fe4-4277-89f9-436f4f6f84d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:20.514381 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f8cfd-5fe4-4277-89f9-436f4f6f84d1 admin admin] PATH_INFO: `/v3/projects/3c1e417367fe41c9943a8b2297742120/users/c80784a1d58a4b9186636b37d186a739/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:20.529467 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f8cfd-5fe4-4277-89f9-436f4f6f84d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c1e417367fe41c9943a8b2297742120, user_id=c80784a1d58a4b9186636b37d186a739, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:20.532187 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f8cfd-5fe4-4277-89f9-436f4f6f84d1 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:20.545370 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1099/2216] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:20 2022] PUT /identity/v3/projects/3c1e417367fe41c9943a8b2297742120/users/c80784a1d58a4b9186636b37d186a739/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:20.546257 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f8cfd-5fe4-4277-89f9-436f4f6f84d1 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:20.547892 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5d8f8cfd-5fe4-4277-89f9-436f4f6f84d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:20.565098 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f05631-8ee6-47ff-a6ae-ee99c85b30fc admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:20.597391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:20.617007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f8cfd-5fe4-4277-89f9-436f4f6f84d1 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:20.622456 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f8c05-ac24-4899-89bf-08b98e5dd74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:20.623373 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f8c05-ac24-4899-89bf-08b98e5dd74f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:20.623658 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f8c05-ac24-4899-89bf-08b98e5dd74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:20.623933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f8c05-ac24-4899-89bf-08b98e5dd74f admin admin] PATH_INFO: `/v3/users/f3413c3709be4c22823c88cf2f208820` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:20.625150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f8c05-ac24-4899-89bf-08b98e5dd74f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3413c3709be4c22823c88cf2f208820)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:20.628146 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f8c05-ac24-4899-89bf-08b98e5dd74f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:20.630269 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3657a9-18f0-4a75-ac27-4a657ecf6631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:20.631837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3657a9-18f0-4a75-ac27-4a657ecf6631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:20.632221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3657a9-18f0-4a75-ac27-4a657ecf6631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:20.632676 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3657a9-18f0-4a75-ac27-4a657ecf6631 admin admin] PATH_INFO: `/v3/users/748ba04206184146a9ae623d5fec2121` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:20.634256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3657a9-18f0-4a75-ac27-4a657ecf6631 admin admin] RBAC: Authorizing `identity:delete_user(user_id=748ba04206184146a9ae623d5fec2121)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:20.637076 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3657a9-18f0-4a75-ac27-4a657ecf6631 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:20.676232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:20.682362 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e70f8c05-ac24-4899-89bf-08b98e5dd74f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3413c3709be4c22823c88cf2f208820'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.687228 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ee3657a9-18f0-4a75-ac27-4a657ecf6631 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '748ba04206184146a9ae623d5fec2121'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.692408 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e70f8c05-ac24-4899-89bf-08b98e5dd74f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3413c3709be4c22823c88cf2f208820'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.693182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e70f8c05-ac24-4899-89bf-08b98e5dd74f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3413c3709be4c22823c88cf2f208820'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.693942 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e70f8c05-ac24-4899-89bf-08b98e5dd74f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3413c3709be4c22823c88cf2f208820'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.695436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ee3657a9-18f0-4a75-ac27-4a657ecf6631 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '748ba04206184146a9ae623d5fec2121'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.702410 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ee3657a9-18f0-4a75-ac27-4a657ecf6631 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '748ba04206184146a9ae623d5fec2121'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.703011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ee3657a9-18f0-4a75-ac27-4a657ecf6631 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '748ba04206184146a9ae623d5fec2121'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.703627 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ee3657a9-18f0-4a75-ac27-4a657ecf6631 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '748ba04206184146a9ae623d5fec2121'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.706823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e70f8c05-ac24-4899-89bf-08b98e5dd74f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3413c3709be4c22823c88cf2f208820'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:20.719591 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1100/2217] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:20 2022] DELETE /identity/v3/users/f3413c3709be4c22823c88cf2f208820 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:20.721374 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e11fb-401b-4180-858d-2f5fd0f3ffc0 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:20.721835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e11fb-401b-4180-858d-2f5fd0f3ffc0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:20.722258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e11fb-401b-4180-858d-2f5fd0f3ffc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:20.722897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1118/2218] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:20 2022] DELETE /identity/v3/users/748ba04206184146a9ae623d5fec2121 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:20.724463 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc419d-51ec-4569-855b-aef2aca71ab5 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:20.724879 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc419d-51ec-4569-855b-aef2aca71ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:20.725253 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc419d-51ec-4569-855b-aef2aca71ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:20.758976 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d17e11fb-401b-4180-858d-2f5fd0f3ffc0 None None] MFA Rules not processed for user `2ec1cd2998844a7d85840ac284083d3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:20.759328 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-20cc419d-51ec-4569-855b-aef2aca71ab5 None None] MFA Rules not processed for user `c80784a1d58a4b9186636b37d186a739`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:20.760317 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d17e11fb-401b-4180-858d-2f5fd0f3ffc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:20.760654 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-20cc419d-51ec-4569-855b-aef2aca71ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:20.847170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1119/2219] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:20.862181 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc419d-51ec-4569-855b-aef2aca71ab5 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:20.863520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1101/2220] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:20 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:20.864727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-20cc419d-51ec-4569-855b-aef2aca71ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:20.877651 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e11fb-401b-4180-858d-2f5fd0f3ffc0 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:20.879562 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d17e11fb-401b-4180-858d-2f5fd0f3ffc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:20.965910 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc419d-51ec-4569-855b-aef2aca71ab5 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:20.970882 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e11fb-401b-4180-858d-2f5fd0f3ffc0 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:20.971662 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107e027-324e-4e4d-9574-e648d58c9ba0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:20.972567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107e027-324e-4e4d-9574-e648d58c9ba0 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:20.973056 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107e027-324e-4e4d-9574-e648d58c9ba0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:20.973549 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107e027-324e-4e4d-9574-e648d58c9ba0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:20.976030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b3fa2-a355-4aec-9a10-3e1cf677b677 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:20.976693 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b3fa2-a355-4aec-9a10-3e1cf677b677 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:20.976978 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b3fa2-a355-4aec-9a10-3e1cf677b677 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:20.977243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b3fa2-a355-4aec-9a10-3e1cf677b677 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:20.984567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107e027-324e-4e4d-9574-e648d58c9ba0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:20.986908 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107e027-324e-4e4d-9574-e648d58c9ba0 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:20.987666 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:20.993038 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b3fa2-a355-4aec-9a10-3e1cf677b677 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:20.994751 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b3fa2-a355-4aec-9a10-3e1cf677b677 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:21.001217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1120/2221] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:36:21.012134 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1102/2222] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:36:21.074000 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107e027-324e-4e4d-9574-e648d58c9ba0 service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:21.076767 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-0107e027-324e-4e4d-9574-e648d58c9ba0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:21.092115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b3fa2-a355-4aec-9a10-3e1cf677b677 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:21.094153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e47b3fa2-a355-4aec-9a10-3e1cf677b677 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:21.162154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:21.191251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107e027-324e-4e4d-9574-e648d58c9ba0 service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:21.192684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b3fa2-a355-4aec-9a10-3e1cf677b677 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:21.197016 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d665a0-0f74-476a-be32-269276501ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:21.198033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d665a0-0f74-476a-be32-269276501ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:21.198260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed172fc4-5c8b-48f1-8ac0-60e0533aadbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:21.198463 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d665a0-0f74-476a-be32-269276501ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:21.199131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d665a0-0f74-476a-be32-269276501ce7 admin admin] PATH_INFO: `/v3/projects/43ebe22b20c94f49bb43c4689891fbd6` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:21.199390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed172fc4-5c8b-48f1-8ac0-60e0533aadbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:21.199692 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed172fc4-5c8b-48f1-8ac0-60e0533aadbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:21.199984 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed172fc4-5c8b-48f1-8ac0-60e0533aadbe admin admin] PATH_INFO: `/v3/projects/a2f412b417e14db885dfe3cfa6059876` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:21.201775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed172fc4-5c8b-48f1-8ac0-60e0533aadbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2f412b417e14db885dfe3cfa6059876)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:21.201899 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d665a0-0f74-476a-be32-269276501ce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43ebe22b20c94f49bb43c4689891fbd6)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:21.203936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d665a0-0f74-476a-be32-269276501ce7 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:21.204080 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed172fc4-5c8b-48f1-8ac0-60e0533aadbe admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:21.262429 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d1d665a0-0f74-476a-be32-269276501ce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43ebe22b20c94f49bb43c4689891fbd6', 'request_id': 'req-d1d665a0-0f74-476a-be32-269276501ce7'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:21.272523 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d1d665a0-0f74-476a-be32-269276501ce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43ebe22b20c94f49bb43c4689891fbd6', 'request_id': 'req-d1d665a0-0f74-476a-be32-269276501ce7'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:21.276683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ed172fc4-5c8b-48f1-8ac0-60e0533aadbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2f412b417e14db885dfe3cfa6059876', 'request_id': 'req-ed172fc4-5c8b-48f1-8ac0-60e0533aadbe'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:21.281475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d1d665a0-0f74-476a-be32-269276501ce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43ebe22b20c94f49bb43c4689891fbd6', 'request_id': 'req-d1d665a0-0f74-476a-be32-269276501ce7'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:21.288534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ed172fc4-5c8b-48f1-8ac0-60e0533aadbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2f412b417e14db885dfe3cfa6059876', 'request_id': 'req-ed172fc4-5c8b-48f1-8ac0-60e0533aadbe'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:21.293131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d1d665a0-0f74-476a-be32-269276501ce7 admin admin] The token cache is being invalidate because project 43ebe22b20c94f49bb43c4689891fbd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:21.293428 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d1d665a0-0f74-476a-be32-269276501ce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:21.294341 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d1d665a0-0f74-476a-be32-269276501ce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:21.295117 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ed172fc4-5c8b-48f1-8ac0-60e0533aadbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2f412b417e14db885dfe3cfa6059876', 'request_id': 'req-ed172fc4-5c8b-48f1-8ac0-60e0533aadbe'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:21.296174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1121/2223] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:21 2022] DELETE /identity/v3/projects/43ebe22b20c94f49bb43c4689891fbd6 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:21.296817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d665a0-0f74-476a-be32-269276501ce7 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:21.298542 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d1d665a0-0f74-476a-be32-269276501ce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:21.315996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ed172fc4-5c8b-48f1-8ac0-60e0533aadbe admin admin] The token cache is being invalidate because project a2f412b417e14db885dfe3cfa6059876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:21.316554 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ed172fc4-5c8b-48f1-8ac0-60e0533aadbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:21.317399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ed172fc4-5c8b-48f1-8ac0-60e0533aadbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:21.322754 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1103/2224] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:21 2022] DELETE /identity/v3/projects/a2f412b417e14db885dfe3cfa6059876 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:36:21.323946 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed172fc4-5c8b-48f1-8ac0-60e0533aadbe admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:21.325639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ed172fc4-5c8b-48f1-8ac0-60e0533aadbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:21.419363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d665a0-0f74-476a-be32-269276501ce7 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:21.468016 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed172fc4-5c8b-48f1-8ac0-60e0533aadbe admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:21.481639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9672c-970b-4f86-976f-7353c1fa9543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:21.484710 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9672c-970b-4f86-976f-7353c1fa9543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:21.484998 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9672c-970b-4f86-976f-7353c1fa9543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:21.485429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9672c-970b-4f86-976f-7353c1fa9543 admin admin] PATH_INFO: `/v3/users/fe41d0ab2d8b4f8d95d8be995833ba0a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:21.489706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9672c-970b-4f86-976f-7353c1fa9543 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe41d0ab2d8b4f8d95d8be995833ba0a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:21.495223 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9672c-970b-4f86-976f-7353c1fa9543 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:21.572133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:21.580171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319dac2-7fcc-47d5-8185-b0e08cad02d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:21.580999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319dac2-7fcc-47d5-8185-b0e08cad02d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:21.581296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319dac2-7fcc-47d5-8185-b0e08cad02d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:21.581713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319dac2-7fcc-47d5-8185-b0e08cad02d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:21.583713 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d319dac2-7fcc-47d5-8185-b0e08cad02d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:21.601888 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c1e9672c-970b-4f86-976f-7353c1fa9543 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe41d0ab2d8b4f8d95d8be995833ba0a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:21.612895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c1e9672c-970b-4f86-976f-7353c1fa9543 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe41d0ab2d8b4f8d95d8be995833ba0a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:21.613600 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c1e9672c-970b-4f86-976f-7353c1fa9543 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe41d0ab2d8b4f8d95d8be995833ba0a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:21.614462 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c1e9672c-970b-4f86-976f-7353c1fa9543 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe41d0ab2d8b4f8d95d8be995833ba0a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:21.637401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:21.640819 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c1e9672c-970b-4f86-976f-7353c1fa9543 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe41d0ab2d8b4f8d95d8be995833ba0a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:21.654859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d319dac2-7fcc-47d5-8185-b0e08cad02d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:21.655069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1104/2225] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:21 2022] DELETE /identity/v3/users/fe41d0ab2d8b4f8d95d8be995833ba0a => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:21.656476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d319dac2-7fcc-47d5-8185-b0e08cad02d2 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:21.668363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1122/2226] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:21.940862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9672c-970b-4f86-976f-7353c1fa9543 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:21.942343 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c1e9672c-970b-4f86-976f-7353c1fa9543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:22.011555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9672c-970b-4f86-976f-7353c1fa9543 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:22.016132 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61d204-9f69-4c12-99e8-6cd37c22f0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:22.017509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61d204-9f69-4c12-99e8-6cd37c22f0bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:22.017838 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61d204-9f69-4c12-99e8-6cd37c22f0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:22.018088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61d204-9f69-4c12-99e8-6cd37c22f0bc admin admin] PATH_INFO: `/v3/projects/44da29c85fa341debb198a0b2a47f79e` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:22.019318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61d204-9f69-4c12-99e8-6cd37c22f0bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=44da29c85fa341debb198a0b2a47f79e)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:22.021477 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61d204-9f69-4c12-99e8-6cd37c22f0bc admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:22.067742 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0f61d204-9f69-4c12-99e8-6cd37c22f0bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44da29c85fa341debb198a0b2a47f79e', 'request_id': 'req-0f61d204-9f69-4c12-99e8-6cd37c22f0bc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:22.074002 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0f61d204-9f69-4c12-99e8-6cd37c22f0bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44da29c85fa341debb198a0b2a47f79e', 'request_id': 'req-0f61d204-9f69-4c12-99e8-6cd37c22f0bc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:22.080110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0f61d204-9f69-4c12-99e8-6cd37c22f0bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44da29c85fa341debb198a0b2a47f79e', 'request_id': 'req-0f61d204-9f69-4c12-99e8-6cd37c22f0bc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:22.094491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0f61d204-9f69-4c12-99e8-6cd37c22f0bc admin admin] The token cache is being invalidate because project 44da29c85fa341debb198a0b2a47f79e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:22.094779 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0f61d204-9f69-4c12-99e8-6cd37c22f0bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:22.096052 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-0f61d204-9f69-4c12-99e8-6cd37c22f0bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:22.097629 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1105/2227] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:21 2022] DELETE /identity/v3/projects/44da29c85fa341debb198a0b2a47f79e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:22.102694 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96f39c-527b-4d0c-9fbd-b1b0ee0ddb35 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:22.103195 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96f39c-527b-4d0c-9fbd-b1b0ee0ddb35 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:22.103291 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96f39c-527b-4d0c-9fbd-b1b0ee0ddb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:22.120381 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-1b96f39c-527b-4d0c-9fbd-b1b0ee0ddb35 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:22.121570 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-1b96f39c-527b-4d0c-9fbd-b1b0ee0ddb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:22.180300 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1123/2228] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:22.188914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b8b4b-bbf9-4691-857e-cccdd245277c None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:22.189205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b8b4b-bbf9-4691-857e-cccdd245277c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:22.189482 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b8b4b-bbf9-4691-857e-cccdd245277c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:22.208258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-972b8b4b-bbf9-4691-857e-cccdd245277c None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:22.209467 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-972b8b4b-bbf9-4691-857e-cccdd245277c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:22.217507 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1106/2229] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:22.222500 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96f39c-527b-4d0c-9fbd-b1b0ee0ddb35 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:22.232438 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96f39c-527b-4d0c-9fbd-b1b0ee0ddb35 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:22.236977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ab2cd-573b-45f1-b0df-2b320a926ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:22.237627 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ab2cd-573b-45f1-b0df-2b320a926ccc admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:22.237907 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ab2cd-573b-45f1-b0df-2b320a926ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:22.238183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ab2cd-573b-45f1-b0df-2b320a926ccc admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:22.238907 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ab2cd-573b-45f1-b0df-2b320a926ccc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:36:22.239251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ab2cd-573b-45f1-b0df-2b320a926ccc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:22.240827 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ab2cd-573b-45f1-b0df-2b320a926ccc admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:22.252780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1124/2230] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:36:22.257844 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b8b4b-bbf9-4691-857e-cccdd245277c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:22.268320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b8b4b-bbf9-4691-857e-cccdd245277c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:22.273316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119fdfd-7515-4863-ac32-e98c4db363e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:22.273969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119fdfd-7515-4863-ac32-e98c4db363e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:22.274277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119fdfd-7515-4863-ac32-e98c4db363e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:22.274562 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119fdfd-7515-4863-ac32-e98c4db363e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:22.275315 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4119fdfd-7515-4863-ac32-e98c4db363e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:22.277269 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4119fdfd-7515-4863-ac32-e98c4db363e6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:22.303198 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1107/2231] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:22 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:22.307347 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ab2cd-573b-45f1-b0df-2b320a926ccc admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:22.318236 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ab2cd-573b-45f1-b0df-2b320a926ccc admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:22.373369 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d593b6-f978-4c8e-8f80-61a784267b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:22.373783 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d593b6-f978-4c8e-8f80-61a784267b7e admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:22.374002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d593b6-f978-4c8e-8f80-61a784267b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:22.374164 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d593b6-f978-4c8e-8f80-61a784267b7e admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:22.374866 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d593b6-f978-4c8e-8f80-61a784267b7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:22.377367 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d593b6-f978-4c8e-8f80-61a784267b7e admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:22.402147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1125/2232] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:22 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:22.406439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119fdfd-7515-4863-ac32-e98c4db363e6 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:22.413190 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:22.417405 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119fdfd-7515-4863-ac32-e98c4db363e6 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:22.423547 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72231ff7-8abd-4cff-bc3e-55f4c69ea029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:22.424276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72231ff7-8abd-4cff-bc3e-55f4c69ea029 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:22.424529 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72231ff7-8abd-4cff-bc3e-55f4c69ea029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:22.425151 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72231ff7-8abd-4cff-bc3e-55f4c69ea029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:22.426140 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72231ff7-8abd-4cff-bc3e-55f4c69ea029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:22.428032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72231ff7-8abd-4cff-bc3e-55f4c69ea029 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:22.439317 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1108/2233] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:22.443392 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d593b6-f978-4c8e-8f80-61a784267b7e admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:22.453509 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d593b6-f978-4c8e-8f80-61a784267b7e admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:22.458776 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7af39-f7ef-40aa-97d2-bb446d942f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:22.459466 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7af39-f7ef-40aa-97d2-bb446d942f29 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:22.459752 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7af39-f7ef-40aa-97d2-bb446d942f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:22.460069 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7af39-f7ef-40aa-97d2-bb446d942f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:22.460764 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7af39-f7ef-40aa-97d2-bb446d942f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:22.462338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7af39-f7ef-40aa-97d2-bb446d942f29 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:22.474122 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1126/2234] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:22.478289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72231ff7-8abd-4cff-bc3e-55f4c69ea029 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:22.488438 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72231ff7-8abd-4cff-bc3e-55f4c69ea029 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:22.493042 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c99f441-15cd-479b-89f6-5085c9349ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:22.493904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99f441-15cd-479b-89f6-5085c9349ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:22.494186 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99f441-15cd-479b-89f6-5085c9349ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:22.494461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99f441-15cd-479b-89f6-5085c9349ccf admin admin] PATH_INFO: `/v3/projects/03288f0649a848fca16eda063bc9817e/users/3a740dea2e494a4d80d1b428a373e930/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:22.512204 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c99f441-15cd-479b-89f6-5085c9349ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=03288f0649a848fca16eda063bc9817e, user_id=3a740dea2e494a4d80d1b428a373e930, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:22.515748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c99f441-15cd-479b-89f6-5085c9349ccf admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:22.532074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1109/2235] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:22 2022] PUT /identity/v3/projects/03288f0649a848fca16eda063bc9817e/users/3a740dea2e494a4d80d1b428a373e930/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:22.570929 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e6731-4e7f-45df-be51-abd641814c97 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:22.571251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e6731-4e7f-45df-be51-abd641814c97 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:22.571527 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e6731-4e7f-45df-be51-abd641814c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:22.596147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-6c6e6731-4e7f-45df-be51-abd641814c97 None None] MFA Rules not processed for user `c80784a1d58a4b9186636b37d186a739`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:22.596147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6c6e6731-4e7f-45df-be51-abd641814c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:22.600767 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b493d6-8769-40b1-bff2-abe8dcc46340 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:22.601165 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b493d6-8769-40b1-bff2-abe8dcc46340 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:22.601568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b493d6-8769-40b1-bff2-abe8dcc46340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:22.636306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-31b493d6-8769-40b1-bff2-abe8dcc46340 None None] MFA Rules not processed for user `3a740dea2e494a4d80d1b428a373e930`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:22.637873 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-31b493d6-8769-40b1-bff2-abe8dcc46340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:22.656142 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:22.676305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1127/2236] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:22.734775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1110/2237] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:36:22.754148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e6731-4e7f-45df-be51-abd641814c97 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:22.755445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6c6e6731-4e7f-45df-be51-abd641814c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:22.853212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e6731-4e7f-45df-be51-abd641814c97 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:22.858518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60436cf-66cd-46fe-87a0-4a92abd25a9f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b7a38b4a7754037a00d11dafc88645b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:22.859357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60436cf-66cd-46fe-87a0-4a92abd25a9f service glance] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:22.860025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60436cf-66cd-46fe-87a0-4a92abd25a9f service glance] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:22.860310 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60436cf-66cd-46fe-87a0-4a92abd25a9f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:22.871916 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60436cf-66cd-46fe-87a0-4a92abd25a9f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:22.874135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60436cf-66cd-46fe-87a0-4a92abd25a9f service glance] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:22.885253 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1128/2238] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:22 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:24.954620 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b493d6-8769-40b1-bff2-abe8dcc46340 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:24.956611 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-31b493d6-8769-40b1-bff2-abe8dcc46340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:25.025682 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b493d6-8769-40b1-bff2-abe8dcc46340 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:25.030357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69716c-49f5-4224-a691-b65797f1bfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:25.031265 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69716c-49f5-4224-a691-b65797f1bfbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:25.031541 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69716c-49f5-4224-a691-b65797f1bfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:25.031815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69716c-49f5-4224-a691-b65797f1bfbf admin admin] PATH_INFO: `/v3/users/1fac7295213842e4933df69eca61c9ee` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:25.032998 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69716c-49f5-4224-a691-b65797f1bfbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fac7295213842e4933df69eca61c9ee)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:25.035558 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69716c-49f5-4224-a691-b65797f1bfbf admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:25.079699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1d69716c-49f5-4224-a691-b65797f1bfbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fac7295213842e4933df69eca61c9ee'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:25.088040 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1d69716c-49f5-4224-a691-b65797f1bfbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fac7295213842e4933df69eca61c9ee'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:25.088642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1d69716c-49f5-4224-a691-b65797f1bfbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fac7295213842e4933df69eca61c9ee'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:25.089233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1d69716c-49f5-4224-a691-b65797f1bfbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fac7295213842e4933df69eca61c9ee'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:25.094549 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:25.103533 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1d69716c-49f5-4224-a691-b65797f1bfbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fac7295213842e4933df69eca61c9ee'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:25.115968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1111/2239] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:24 2022] DELETE /identity/v3/users/1fac7295213842e4933df69eca61c9ee => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:25.120704 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03568b-f642-4085-b786-fa2159eae8b6 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:25.121167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03568b-f642-4085-b786-fa2159eae8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:25.121258 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03568b-f642-4085-b786-fa2159eae8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:25.139643 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-ef03568b-f642-4085-b786-fa2159eae8b6 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:25.140860 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ef03568b-f642-4085-b786-fa2159eae8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:25.206288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1129/2240] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:25.215992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69716c-49f5-4224-a691-b65797f1bfbf admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:25.217593 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1d69716c-49f5-4224-a691-b65797f1bfbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:25.288033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69716c-49f5-4224-a691-b65797f1bfbf admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:25.291862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84634792-d35d-4489-9ba6-0012877d74da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:25.292600 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84634792-d35d-4489-9ba6-0012877d74da service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:25.292841 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84634792-d35d-4489-9ba6-0012877d74da service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:25.293158 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84634792-d35d-4489-9ba6-0012877d74da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:25.308762 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84634792-d35d-4489-9ba6-0012877d74da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:25.308762 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84634792-d35d-4489-9ba6-0012877d74da service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:25.318316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1112/2241] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:25.706217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03568b-f642-4085-b786-fa2159eae8b6 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:25.707793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ef03568b-f642-4085-b786-fa2159eae8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:25.719936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03568b-f642-4085-b786-fa2159eae8b6 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:25.725053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03727362-d86e-47a3-bfea-0835f8c6d8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:25.726036 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03727362-d86e-47a3-bfea-0835f8c6d8c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:25.726202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03727362-d86e-47a3-bfea-0835f8c6d8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:25.726474 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03727362-d86e-47a3-bfea-0835f8c6d8c0 admin admin] PATH_INFO: `/v3/projects/fc295a123da6438ea9925f091301dcaf` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:25.727726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03727362-d86e-47a3-bfea-0835f8c6d8c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc295a123da6438ea9925f091301dcaf)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:25.729732 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03727362-d86e-47a3-bfea-0835f8c6d8c0 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:25.790557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-03727362-d86e-47a3-bfea-0835f8c6d8c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc295a123da6438ea9925f091301dcaf', 'request_id': 'req-03727362-d86e-47a3-bfea-0835f8c6d8c0'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:25.797390 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-03727362-d86e-47a3-bfea-0835f8c6d8c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc295a123da6438ea9925f091301dcaf', 'request_id': 'req-03727362-d86e-47a3-bfea-0835f8c6d8c0'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:25.810130 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-03727362-d86e-47a3-bfea-0835f8c6d8c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc295a123da6438ea9925f091301dcaf', 'request_id': 'req-03727362-d86e-47a3-bfea-0835f8c6d8c0'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:25.822116 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-03727362-d86e-47a3-bfea-0835f8c6d8c0 admin admin] The token cache is being invalidate because project fc295a123da6438ea9925f091301dcaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:25.822385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-03727362-d86e-47a3-bfea-0835f8c6d8c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:25.823088 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-03727362-d86e-47a3-bfea-0835f8c6d8c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:25.824694 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1130/2242] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:25 2022] DELETE /identity/v3/projects/fc295a123da6438ea9925f091301dcaf => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:25.831169 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7bc47-9641-4d0b-b072-68357f8d9518 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:25.831511 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7bc47-9641-4d0b-b072-68357f8d9518 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:25.831818 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7bc47-9641-4d0b-b072-68357f8d9518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:25.850012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-b5c7bc47-9641-4d0b-b072-68357f8d9518 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:25.851252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b5c7bc47-9641-4d0b-b072-68357f8d9518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:25.911677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1113/2243] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:25.920828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05347505-7b36-4226-ac49-ea99ecedd259 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:25.921116 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05347505-7b36-4226-ac49-ea99ecedd259 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:25.921396 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05347505-7b36-4226-ac49-ea99ecedd259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:25.941034 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-05347505-7b36-4226-ac49-ea99ecedd259 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:25.942233 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-05347505-7b36-4226-ac49-ea99ecedd259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:25.950155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1131/2244] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:25.954541 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7bc47-9641-4d0b-b072-68357f8d9518 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:25.964289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7bc47-9641-4d0b-b072-68357f8d9518 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:25.968916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe847d9-e719-473d-889b-b4991f190553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:25.969568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe847d9-e719-473d-889b-b4991f190553 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:25.969852 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe847d9-e719-473d-889b-b4991f190553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:25.970179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe847d9-e719-473d-889b-b4991f190553 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:25.971478 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe847d9-e719-473d-889b-b4991f190553 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:36:25.971785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe847d9-e719-473d-889b-b4991f190553 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:25.973382 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe847d9-e719-473d-889b-b4991f190553 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:25.984906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1114/2245] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:36:25.989803 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05347505-7b36-4226-ac49-ea99ecedd259 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:25.999083 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05347505-7b36-4226-ac49-ea99ecedd259 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:26.003659 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53113e8f-4ff0-4c41-8b1a-f16ed59eed3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:26.004299 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53113e8f-4ff0-4c41-8b1a-f16ed59eed3a admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:26.004580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53113e8f-4ff0-4c41-8b1a-f16ed59eed3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:26.004853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53113e8f-4ff0-4c41-8b1a-f16ed59eed3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:26.005568 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53113e8f-4ff0-4c41-8b1a-f16ed59eed3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:26.007477 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53113e8f-4ff0-4c41-8b1a-f16ed59eed3a admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:26.024046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:26.040811 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1132/2246] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:25 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:26.045059 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe847d9-e719-473d-889b-b4991f190553 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:26.056078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe847d9-e719-473d-889b-b4991f190553 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:26.110850 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7849e-0d55-45b6-bf79-26cf09a4d453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:26.112120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7849e-0d55-45b6-bf79-26cf09a4d453 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:26.112404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7849e-0d55-45b6-bf79-26cf09a4d453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:26.112681 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7849e-0d55-45b6-bf79-26cf09a4d453 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:26.113384 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7849e-0d55-45b6-bf79-26cf09a4d453 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:26.115340 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7849e-0d55-45b6-bf79-26cf09a4d453 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:26.129088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:26.135647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1115/2247] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:26 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:26.139900 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53113e8f-4ff0-4c41-8b1a-f16ed59eed3a admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:26.152334 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53113e8f-4ff0-4c41-8b1a-f16ed59eed3a admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:26.157858 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587f30e-0fed-4ddc-9cc1-969add3e4f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:26.159282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587f30e-0fed-4ddc-9cc1-969add3e4f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:26.159604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587f30e-0fed-4ddc-9cc1-969add3e4f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:26.159879 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587f30e-0fed-4ddc-9cc1-969add3e4f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:26.160620 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587f30e-0fed-4ddc-9cc1-969add3e4f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:26.162238 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587f30e-0fed-4ddc-9cc1-969add3e4f9f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:26.174555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1133/2248] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:36:26.177980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7849e-0d55-45b6-bf79-26cf09a4d453 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:26.188046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7849e-0d55-45b6-bf79-26cf09a4d453 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:26.192720 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff185f-df43-4c47-b54e-0335e03579c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:26.193372 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff185f-df43-4c47-b54e-0335e03579c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:26.193651 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff185f-df43-4c47-b54e-0335e03579c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:26.193927 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff185f-df43-4c47-b54e-0335e03579c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:26.194651 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff185f-df43-4c47-b54e-0335e03579c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:26.196267 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff185f-df43-4c47-b54e-0335e03579c4 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:26.206185 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1116/2249] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:26.210262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587f30e-0fed-4ddc-9cc1-969add3e4f9f admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:26.220361 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587f30e-0fed-4ddc-9cc1-969add3e4f9f admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:26.225346 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb2fc5-7dff-4f18-9d84-5f785b0e1518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:26.226214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb2fc5-7dff-4f18-9d84-5f785b0e1518 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:26.226495 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb2fc5-7dff-4f18-9d84-5f785b0e1518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:26.226770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb2fc5-7dff-4f18-9d84-5f785b0e1518 admin admin] PATH_INFO: `/v3/projects/83a29cbfb7484b76a9131a3b81832c62/users/6502e3ffa3b84333a00ea5f712767d61/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:26.241850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb2fc5-7dff-4f18-9d84-5f785b0e1518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83a29cbfb7484b76a9131a3b81832c62, user_id=6502e3ffa3b84333a00ea5f712767d61, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:26.244563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb2fc5-7dff-4f18-9d84-5f785b0e1518 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:26.260040 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1134/2250] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:26 2022] PUT /identity/v3/projects/83a29cbfb7484b76a9131a3b81832c62/users/6502e3ffa3b84333a00ea5f712767d61/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:36:26.264893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc681f4b-2fa3-4885-ab46-69739943b5e5 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:26.265183 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc681f4b-2fa3-4885-ab46-69739943b5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:26.265463 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc681f4b-2fa3-4885-ab46-69739943b5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:26.283826 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-fc681f4b-2fa3-4885-ab46-69739943b5e5 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:26.285163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fc681f4b-2fa3-4885-ab46-69739943b5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:26.347130 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1117/2251] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:26.356793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb2fc5-7dff-4f18-9d84-5f785b0e1518 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:26.358672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-62fb2fc5-7dff-4f18-9d84-5f785b0e1518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:26.416051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb2fc5-7dff-4f18-9d84-5f785b0e1518 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:26.419808 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409cf980-90b9-4ca4-9eee-87e355948535 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:26.420468 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409cf980-90b9-4ca4-9eee-87e355948535 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:26.420752 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409cf980-90b9-4ca4-9eee-87e355948535 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:26.421026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409cf980-90b9-4ca4-9eee-87e355948535 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:26.431468 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409cf980-90b9-4ca4-9eee-87e355948535 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:26.432898 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409cf980-90b9-4ca4-9eee-87e355948535 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:26.443810 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1135/2252] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:31.734083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714d00c-1e0d-42c9-82a6-8fdbed63a805 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:31.734083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714d00c-1e0d-42c9-82a6-8fdbed63a805 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:31.734882 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714d00c-1e0d-42c9-82a6-8fdbed63a805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:31.792299 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-9714d00c-1e0d-42c9-82a6-8fdbed63a805 None None] MFA Rules not processed for user `6502e3ffa3b84333a00ea5f712767d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:31.811368 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9714d00c-1e0d-42c9-82a6-8fdbed63a805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:31.995772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1118/2253] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:36:31.996504 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409cf980-90b9-4ca4-9eee-87e355948535 service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:32.031842 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409cf980-90b9-4ca4-9eee-87e355948535 service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:32.038545 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11bee8-ce2e-46b0-afd5-c0677d162634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:32.052185 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11bee8-ce2e-46b0-afd5-c0677d162634 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:32.052604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11bee8-ce2e-46b0-afd5-c0677d162634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:32.052967 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11bee8-ce2e-46b0-afd5-c0677d162634 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:32.053767 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11bee8-ce2e-46b0-afd5-c0677d162634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:32.056259 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11bee8-ce2e-46b0-afd5-c0677d162634 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:32.093738 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1136/2254] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:31 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:36:32.104288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714d00c-1e0d-42c9-82a6-8fdbed63a805 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:32.118088 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714d00c-1e0d-42c9-82a6-8fdbed63a805 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:32.252617 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446c729-bcfa-44a6-869d-f5689743bf5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:32.253322 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446c729-bcfa-44a6-869d-f5689743bf5a admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:32.253608 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446c729-bcfa-44a6-869d-f5689743bf5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:32.253886 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446c729-bcfa-44a6-869d-f5689743bf5a admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:32.254592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446c729-bcfa-44a6-869d-f5689743bf5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:32.256782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446c729-bcfa-44a6-869d-f5689743bf5a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:32.286454 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1119/2255] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:32 2022] POST /identity/v3/users => generated 295 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:32.290686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11bee8-ce2e-46b0-afd5-c0677d162634 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:32.306266 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11bee8-ce2e-46b0-afd5-c0677d162634 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:32.311809 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a8ca2-4d91-4122-8717-9fdae66e1673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:32.312492 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a8ca2-4d91-4122-8717-9fdae66e1673 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:32.312957 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a8ca2-4d91-4122-8717-9fdae66e1673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:32.313372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a8ca2-4d91-4122-8717-9fdae66e1673 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:32.314683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a8ca2-4d91-4122-8717-9fdae66e1673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:32.316848 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a8ca2-4d91-4122-8717-9fdae66e1673 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:32.330093 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1137/2256] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:32.333919 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446c729-bcfa-44a6-869d-f5689743bf5a admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:32.346174 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446c729-bcfa-44a6-869d-f5689743bf5a admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:32.358031 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d027f-9d90-4f59-80a3-5c21b45aae5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:32.358969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d027f-9d90-4f59-80a3-5c21b45aae5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:32.359246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d027f-9d90-4f59-80a3-5c21b45aae5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:32.359541 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d027f-9d90-4f59-80a3-5c21b45aae5a admin admin] PATH_INFO: `/v3/projects/8198771fe32f4a978926e04d278220ba/users/a1f5c428bab24681b4b91bc04f200f7b/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:32.363991 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:32.386041 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d027f-9d90-4f59-80a3-5c21b45aae5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8198771fe32f4a978926e04d278220ba, user_id=a1f5c428bab24681b4b91bc04f200f7b, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:32.389810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d027f-9d90-4f59-80a3-5c21b45aae5a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:32.414427 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1120/2257] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:32 2022] PUT /identity/v3/projects/8198771fe32f4a978926e04d278220ba/users/a1f5c428bab24681b4b91bc04f200f7b/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:32.416383 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a8ca2-4d91-4122-8717-9fdae66e1673 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:32.430005 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a8ca2-4d91-4122-8717-9fdae66e1673 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:32.544844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d92aa3-ec1e-4de0-a72e-47257c4ffde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:32.545789 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d92aa3-ec1e-4de0-a72e-47257c4ffde1 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:32.546084 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d92aa3-ec1e-4de0-a72e-47257c4ffde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:32.546771 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d92aa3-ec1e-4de0-a72e-47257c4ffde1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:32.547601 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d92aa3-ec1e-4de0-a72e-47257c4ffde1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:32.549648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d92aa3-ec1e-4de0-a72e-47257c4ffde1 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:32.565061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1138/2258] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:32.566872 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d027f-9d90-4f59-80a3-5c21b45aae5a admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:32.585567 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d027f-9d90-4f59-80a3-5c21b45aae5a admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:32.597124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5db96-7c49-48b5-b6fe-96aa5b42dee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:32.599644 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5db96-7c49-48b5-b6fe-96aa5b42dee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:32.600213 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5db96-7c49-48b5-b6fe-96aa5b42dee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:32.600789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5db96-7c49-48b5-b6fe-96aa5b42dee6 admin admin] PATH_INFO: `/v3/projects/8198771fe32f4a978926e04d278220ba/users/a1f5c428bab24681b4b91bc04f200f7b/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:32.604910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5db96-7c49-48b5-b6fe-96aa5b42dee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8198771fe32f4a978926e04d278220ba, user_id=a1f5c428bab24681b4b91bc04f200f7b, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:32.612736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5db96-7c49-48b5-b6fe-96aa5b42dee6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:32.642168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1121/2259] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:32 2022] PUT /identity/v3/projects/8198771fe32f4a978926e04d278220ba/users/a1f5c428bab24681b4b91bc04f200f7b/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:32.818745 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d92aa3-ec1e-4de0-a72e-47257c4ffde1 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:32.820724 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-67d92aa3-ec1e-4de0-a72e-47257c4ffde1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:32.844159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d92aa3-ec1e-4de0-a72e-47257c4ffde1 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:32.883865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:32.949496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4fc19-e718-4b5e-bf1e-e2f3de3aa6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:32.950038 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4fc19-e718-4b5e-bf1e-e2f3de3aa6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:32.950534 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4fc19-e718-4b5e-bf1e-e2f3de3aa6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:32.950895 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4fc19-e718-4b5e-bf1e-e2f3de3aa6c0 admin admin] PATH_INFO: `/v3/users/af83c0aaa726410fbb1e4e7ce61d0c25` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:32.952361 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4fc19-e718-4b5e-bf1e-e2f3de3aa6c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af83c0aaa726410fbb1e4e7ce61d0c25)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:32.954425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4fc19-e718-4b5e-bf1e-e2f3de3aa6c0 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:33.044668 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0bc4fc19-e718-4b5e-bf1e-e2f3de3aa6c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af83c0aaa726410fbb1e4e7ce61d0c25'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:33.055000 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0bc4fc19-e718-4b5e-bf1e-e2f3de3aa6c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af83c0aaa726410fbb1e4e7ce61d0c25'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:33.067509 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0bc4fc19-e718-4b5e-bf1e-e2f3de3aa6c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af83c0aaa726410fbb1e4e7ce61d0c25'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:33.069425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0bc4fc19-e718-4b5e-bf1e-e2f3de3aa6c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af83c0aaa726410fbb1e4e7ce61d0c25'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:33.070108 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0bc4fc19-e718-4b5e-bf1e-e2f3de3aa6c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af83c0aaa726410fbb1e4e7ce61d0c25'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:33.093382 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1139/2260] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:32 2022] DELETE /identity/v3/users/af83c0aaa726410fbb1e4e7ce61d0c25 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:33.098062 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daea25a-2482-4afd-9f8f-9b8dfff37974 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:33.098563 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daea25a-2482-4afd-9f8f-9b8dfff37974 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:33.100501 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daea25a-2482-4afd-9f8f-9b8dfff37974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:33.125610 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4daea25a-2482-4afd-9f8f-9b8dfff37974 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:33.128407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4daea25a-2482-4afd-9f8f-9b8dfff37974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:33.289807 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1122/2261] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:33.328668 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4fc19-e718-4b5e-bf1e-e2f3de3aa6c0 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:33.331688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-0bc4fc19-e718-4b5e-bf1e-e2f3de3aa6c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:33.430304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4fc19-e718-4b5e-bf1e-e2f3de3aa6c0 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:33.435252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c65edc-e6b9-4654-8ac8-97b042f2c83a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:33.436311 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c65edc-e6b9-4654-8ac8-97b042f2c83a service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:33.436788 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c65edc-e6b9-4654-8ac8-97b042f2c83a service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:33.437278 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c65edc-e6b9-4654-8ac8-97b042f2c83a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:33.451400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c65edc-e6b9-4654-8ac8-97b042f2c83a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:33.452969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c65edc-e6b9-4654-8ac8-97b042f2c83a service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:33.491002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1140/2262] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:36:34.243054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daea25a-2482-4afd-9f8f-9b8dfff37974 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:34.245416 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4daea25a-2482-4afd-9f8f-9b8dfff37974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:34.299139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daea25a-2482-4afd-9f8f-9b8dfff37974 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:34.310167 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589a011-7fea-47d6-8917-ec4937ea63f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:34.311178 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589a011-7fea-47d6-8917-ec4937ea63f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:34.311643 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589a011-7fea-47d6-8917-ec4937ea63f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:34.312064 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589a011-7fea-47d6-8917-ec4937ea63f1 admin admin] PATH_INFO: `/v3/projects/4c5bfa264f5746fa98c845ae8bd953eb` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:34.313813 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589a011-7fea-47d6-8917-ec4937ea63f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c5bfa264f5746fa98c845ae8bd953eb)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:34.316970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589a011-7fea-47d6-8917-ec4937ea63f1 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:34.432106 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c589a011-7fea-47d6-8917-ec4937ea63f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c5bfa264f5746fa98c845ae8bd953eb', 'request_id': 'req-c589a011-7fea-47d6-8917-ec4937ea63f1'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:34.444462 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c589a011-7fea-47d6-8917-ec4937ea63f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c5bfa264f5746fa98c845ae8bd953eb', 'request_id': 'req-c589a011-7fea-47d6-8917-ec4937ea63f1'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:34.465670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c589a011-7fea-47d6-8917-ec4937ea63f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c5bfa264f5746fa98c845ae8bd953eb', 'request_id': 'req-c589a011-7fea-47d6-8917-ec4937ea63f1'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:34.489132 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c589a011-7fea-47d6-8917-ec4937ea63f1 admin admin] The token cache is being invalidate because project 4c5bfa264f5746fa98c845ae8bd953eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:34.489302 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c589a011-7fea-47d6-8917-ec4937ea63f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:34.490330 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c589a011-7fea-47d6-8917-ec4937ea63f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:34.493397 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1123/2263] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:34 2022] DELETE /identity/v3/projects/4c5bfa264f5746fa98c845ae8bd953eb => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:34.499468 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c65edc-e6b9-4654-8ac8-97b042f2c83a service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:34.517998 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c65edc-e6b9-4654-8ac8-97b042f2c83a service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:34.611555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:34.647858 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fba30b-ede3-4d11-8f2d-03c40a77563f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:34.648889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fba30b-ede3-4d11-8f2d-03c40a77563f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:34.649399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fba30b-ede3-4d11-8f2d-03c40a77563f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:34.651701 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fba30b-ede3-4d11-8f2d-03c40a77563f admin admin] PATH_INFO: `/v3/users/2ec1cd2998844a7d85840ac284083d3b` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:34.655136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fba30b-ede3-4d11-8f2d-03c40a77563f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ec1cd2998844a7d85840ac284083d3b)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:34.657110 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fba30b-ede3-4d11-8f2d-03c40a77563f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:34.764720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-46fba30b-ede3-4d11-8f2d-03c40a77563f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ec1cd2998844a7d85840ac284083d3b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:34.784268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-46fba30b-ede3-4d11-8f2d-03c40a77563f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ec1cd2998844a7d85840ac284083d3b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:34.797234 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-46fba30b-ede3-4d11-8f2d-03c40a77563f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ec1cd2998844a7d85840ac284083d3b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:34.801398 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-46fba30b-ede3-4d11-8f2d-03c40a77563f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ec1cd2998844a7d85840ac284083d3b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:34.802048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-46fba30b-ede3-4d11-8f2d-03c40a77563f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ec1cd2998844a7d85840ac284083d3b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:34.844738 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1141/2264] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:34 2022] DELETE /identity/v3/users/2ec1cd2998844a7d85840ac284083d3b => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:36.054368 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589a011-7fea-47d6-8917-ec4937ea63f1 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:36.056200 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c589a011-7fea-47d6-8917-ec4937ea63f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:36.114688 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589a011-7fea-47d6-8917-ec4937ea63f1 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:36.327390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:36.365495 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d80e4c-f4b1-4c4a-bc31-834b1149896f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:36.366928 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d80e4c-f4b1-4c4a-bc31-834b1149896f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:36.367555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d80e4c-f4b1-4c4a-bc31-834b1149896f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:36.368068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d80e4c-f4b1-4c4a-bc31-834b1149896f admin admin] PATH_INFO: `/v3/projects/690e437c529c4ec59f8d1087fd970b47` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:36.369804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d80e4c-f4b1-4c4a-bc31-834b1149896f admin admin] RBAC: Authorizing `identity:delete_project(project_id=690e437c529c4ec59f8d1087fd970b47)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:36.373126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d80e4c-f4b1-4c4a-bc31-834b1149896f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:36.499968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b5d80e4c-f4b1-4c4a-bc31-834b1149896f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '690e437c529c4ec59f8d1087fd970b47', 'request_id': 'req-b5d80e4c-f4b1-4c4a-bc31-834b1149896f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:36.512908 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b5d80e4c-f4b1-4c4a-bc31-834b1149896f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '690e437c529c4ec59f8d1087fd970b47', 'request_id': 'req-b5d80e4c-f4b1-4c4a-bc31-834b1149896f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:36.525650 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b5d80e4c-f4b1-4c4a-bc31-834b1149896f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '690e437c529c4ec59f8d1087fd970b47', 'request_id': 'req-b5d80e4c-f4b1-4c4a-bc31-834b1149896f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:36.596775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b5d80e4c-f4b1-4c4a-bc31-834b1149896f admin admin] The token cache is being invalidate because project 690e437c529c4ec59f8d1087fd970b47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:36.597064 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b5d80e4c-f4b1-4c4a-bc31-834b1149896f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:36.600086 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b5d80e4c-f4b1-4c4a-bc31-834b1149896f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:36.608842 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527b614-fa56-4f7c-a46f-7e601a3bf63c None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:36.609430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1124/2265] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:36 2022] DELETE /identity/v3/projects/690e437c529c4ec59f8d1087fd970b47 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:36.609953 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527b614-fa56-4f7c-a46f-7e601a3bf63c None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:36.610618 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527b614-fa56-4f7c-a46f-7e601a3bf63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:36.648283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-b527b614-fa56-4f7c-a46f-7e601a3bf63c None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:36.649818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b527b614-fa56-4f7c-a46f-7e601a3bf63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:36.836907 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1142/2266] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:36:36.848335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315e669-89e5-47e7-a337-305e74ef6b3e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:36.848598 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315e669-89e5-47e7-a337-305e74ef6b3e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:36.849410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315e669-89e5-47e7-a337-305e74ef6b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:36.898972 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-2315e669-89e5-47e7-a337-305e74ef6b3e None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:36.900851 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-2315e669-89e5-47e7-a337-305e74ef6b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:36.916840 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1125/2267] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:36.918028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527b614-fa56-4f7c-a46f-7e601a3bf63c None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:36.939237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527b614-fa56-4f7c-a46f-7e601a3bf63c None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:36.945889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491adea-cbfc-42ef-954d-36f028f0a46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:36.947008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491adea-cbfc-42ef-954d-36f028f0a46f admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:36.947512 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491adea-cbfc-42ef-954d-36f028f0a46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:36.947999 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491adea-cbfc-42ef-954d-36f028f0a46f admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:36.949214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491adea-cbfc-42ef-954d-36f028f0a46f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:36:36.949682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491adea-cbfc-42ef-954d-36f028f0a46f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:36.952418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491adea-cbfc-42ef-954d-36f028f0a46f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:36.978502 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315e669-89e5-47e7-a337-305e74ef6b3e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:36.985815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1143/2268] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:36:36.996315 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315e669-89e5-47e7-a337-305e74ef6b3e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:37.004554 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfde189-2b86-4520-92c2-467c7c08a685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:37.005678 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfde189-2b86-4520-92c2-467c7c08a685 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:37.006163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfde189-2b86-4520-92c2-467c7c08a685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:37.006686 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfde189-2b86-4520-92c2-467c7c08a685 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:37.008001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfde189-2b86-4520-92c2-467c7c08a685 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:37.011246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfde189-2b86-4520-92c2-467c7c08a685 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:37.061037 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491adea-cbfc-42ef-954d-36f028f0a46f admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:37.067745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1126/2269] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:36 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:37.077195 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491adea-cbfc-42ef-954d-36f028f0a46f admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:37.166562 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:37.181821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a990f-e497-4b63-a63d-337fd391cb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:37.182654 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a990f-e497-4b63-a63d-337fd391cb32 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:37.182963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a990f-e497-4b63-a63d-337fd391cb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:37.183256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a990f-e497-4b63-a63d-337fd391cb32 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:37.184256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a990f-e497-4b63-a63d-337fd391cb32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:37.186475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a990f-e497-4b63-a63d-337fd391cb32 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:37.224731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1144/2270] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:37 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:37.229443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfde189-2b86-4520-92c2-467c7c08a685 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:37.244827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfde189-2b86-4520-92c2-467c7c08a685 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:37.252879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd943d4f-ac99-426e-9a50-4e73197e61d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:37.254154 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd943d4f-ac99-426e-9a50-4e73197e61d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:37.254790 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd943d4f-ac99-426e-9a50-4e73197e61d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:37.255445 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd943d4f-ac99-426e-9a50-4e73197e61d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:37.256962 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd943d4f-ac99-426e-9a50-4e73197e61d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:37.259318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd943d4f-ac99-426e-9a50-4e73197e61d8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:37.287762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a990f-e497-4b63-a63d-337fd391cb32 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:37.288153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1127/2271] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:37.317585 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a990f-e497-4b63-a63d-337fd391cb32 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:37.337945 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319f2c9-6ba2-4a76-8e49-8ef615242440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:37.339181 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319f2c9-6ba2-4a76-8e49-8ef615242440 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:37.339868 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319f2c9-6ba2-4a76-8e49-8ef615242440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:37.340589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319f2c9-6ba2-4a76-8e49-8ef615242440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:37.342023 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319f2c9-6ba2-4a76-8e49-8ef615242440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:37.346041 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319f2c9-6ba2-4a76-8e49-8ef615242440 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:37.388410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd943d4f-ac99-426e-9a50-4e73197e61d8 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:37.395498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1145/2272] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:37.406842 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd943d4f-ac99-426e-9a50-4e73197e61d8 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:37.424510 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9e10c-f904-4595-93e6-41fb37b015cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:37.425683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9e10c-f904-4595-93e6-41fb37b015cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:37.426148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9e10c-f904-4595-93e6-41fb37b015cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:37.426681 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9e10c-f904-4595-93e6-41fb37b015cc admin admin] PATH_INFO: `/v3/projects/7b1e02409208460893f61eb01b2cabb4/users/c66283b368a049939a52d554b489dccf/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:37.447790 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9e10c-f904-4595-93e6-41fb37b015cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1e02409208460893f61eb01b2cabb4, user_id=c66283b368a049939a52d554b489dccf, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:37.451808 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9e10c-f904-4595-93e6-41fb37b015cc admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:37.481233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1128/2273] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:37 2022] PUT /identity/v3/projects/7b1e02409208460893f61eb01b2cabb4/users/c66283b368a049939a52d554b489dccf/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:37.491338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd1cec-36cd-4221-bfe5-8c60234a09d1 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:37.491651 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd1cec-36cd-4221-bfe5-8c60234a09d1 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:37.491938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd1cec-36cd-4221-bfe5-8c60234a09d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:37.539480 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-9fbd1cec-36cd-4221-bfe5-8c60234a09d1 None None] MFA Rules not processed for user `c66283b368a049939a52d554b489dccf`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:37.540819 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9fbd1cec-36cd-4221-bfe5-8c60234a09d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:37.728112 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1146/2274] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:37.732680 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9e10c-f904-4595-93e6-41fb37b015cc admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:37.871099 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9e10c-f904-4595-93e6-41fb37b015cc admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:37.882777 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10037a-125c-45b9-822c-6e5bcd5e4e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:37.883528 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10037a-125c-45b9-822c-6e5bcd5e4e64 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:37.884045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10037a-125c-45b9-822c-6e5bcd5e4e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:37.884331 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10037a-125c-45b9-822c-6e5bcd5e4e64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:37.885060 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10037a-125c-45b9-822c-6e5bcd5e4e64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:37.887277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10037a-125c-45b9-822c-6e5bcd5e4e64 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:37.937841 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1129/2275] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:37 2022] POST /identity/v3/projects => generated 373 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:37.947475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd1cec-36cd-4221-bfe5-8c60234a09d1 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:37.976003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd1cec-36cd-4221-bfe5-8c60234a09d1 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:38.116980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9a630-73b0-4cf9-9400-e3fb9cb16c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:38.117852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9a630-73b0-4cf9-9400-e3fb9cb16c7e admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:38.118001 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9a630-73b0-4cf9-9400-e3fb9cb16c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:38.118286 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9a630-73b0-4cf9-9400-e3fb9cb16c7e admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:38.119262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9a630-73b0-4cf9-9400-e3fb9cb16c7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:38.121997 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9a630-73b0-4cf9-9400-e3fb9cb16c7e admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:38.159397 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1147/2276] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:37 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:38.159855 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10037a-125c-45b9-822c-6e5bcd5e4e64 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:38.172378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:38.183431 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10037a-125c-45b9-822c-6e5bcd5e4e64 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:38.193879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4245163-c4cf-4143-b04c-3471f9ef38eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:38.194570 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4245163-c4cf-4143-b04c-3471f9ef38eb admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:38.194855 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4245163-c4cf-4143-b04c-3471f9ef38eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:38.195397 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4245163-c4cf-4143-b04c-3471f9ef38eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:38.196143 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4245163-c4cf-4143-b04c-3471f9ef38eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:38.200805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4245163-c4cf-4143-b04c-3471f9ef38eb admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:38.222084 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1130/2277] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:36:38.227503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9a630-73b0-4cf9-9400-e3fb9cb16c7e admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:38.247348 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9a630-73b0-4cf9-9400-e3fb9cb16c7e admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:38.255780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e82874-f452-49ff-8094-6e25b9fc980b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:38.257682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e82874-f452-49ff-8094-6e25b9fc980b admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:38.258020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e82874-f452-49ff-8094-6e25b9fc980b admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:38.258347 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e82874-f452-49ff-8094-6e25b9fc980b admin admin] PATH_INFO: `/v3/projects/cee1ea0a153340ddbbd3a8bbef497162/users/cb90f2a4013c4d78a49615189cc8ff56/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:38.288002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e82874-f452-49ff-8094-6e25b9fc980b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee1ea0a153340ddbbd3a8bbef497162, user_id=cb90f2a4013c4d78a49615189cc8ff56, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:38.292968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e82874-f452-49ff-8094-6e25b9fc980b admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:38.319198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1148/2278] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:38 2022] PUT /identity/v3/projects/cee1ea0a153340ddbbd3a8bbef497162/users/cb90f2a4013c4d78a49615189cc8ff56/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:38.321867 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4245163-c4cf-4143-b04c-3471f9ef38eb admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:38.334749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4245163-c4cf-4143-b04c-3471f9ef38eb admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:38.427425 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebc0ee-cc20-4b77-8176-beb9e012109c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:38.428140 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebc0ee-cc20-4b77-8176-beb9e012109c admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:38.428731 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebc0ee-cc20-4b77-8176-beb9e012109c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:38.429154 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebc0ee-cc20-4b77-8176-beb9e012109c admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:38.429912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebc0ee-cc20-4b77-8176-beb9e012109c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:38.431808 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebc0ee-cc20-4b77-8176-beb9e012109c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:38.455761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1131/2279] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:38.459923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e82874-f452-49ff-8094-6e25b9fc980b admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:38.475251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e82874-f452-49ff-8094-6e25b9fc980b admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:38.484349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749c063-c079-4ad3-9642-4d9499507acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:38.485310 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749c063-c079-4ad3-9642-4d9499507acb admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:38.486624 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749c063-c079-4ad3-9642-4d9499507acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:38.486624 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749c063-c079-4ad3-9642-4d9499507acb admin admin] PATH_INFO: `/v3/projects/cee1ea0a153340ddbbd3a8bbef497162/users/cb90f2a4013c4d78a49615189cc8ff56/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:38.490135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9749c063-c079-4ad3-9642-4d9499507acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee1ea0a153340ddbbd3a8bbef497162, user_id=cb90f2a4013c4d78a49615189cc8ff56, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:38.492580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9749c063-c079-4ad3-9642-4d9499507acb admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:38.524330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1149/2280] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:38 2022] PUT /identity/v3/projects/cee1ea0a153340ddbbd3a8bbef497162/users/cb90f2a4013c4d78a49615189cc8ff56/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:38.524874 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd45f4-8f2f-4c41-b85d-08ef897627ee None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:38.525403 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd45f4-8f2f-4c41-b85d-08ef897627ee None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:38.526232 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd45f4-8f2f-4c41-b85d-08ef897627ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:38.571656 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-b5cd45f4-8f2f-4c41-b85d-08ef897627ee None None] MFA Rules not processed for user `cb90f2a4013c4d78a49615189cc8ff56`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:38.573759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b5cd45f4-8f2f-4c41-b85d-08ef897627ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:38.683786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1132/2281] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:36:38.697687 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749c063-c079-4ad3-9642-4d9499507acb admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:38.700304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9749c063-c079-4ad3-9642-4d9499507acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:38.717524 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:38.805651 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749c063-c079-4ad3-9642-4d9499507acb admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:38.811247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a5b95-3806-4648-8a78-21419a04c8dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:38.812156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a5b95-3806-4648-8a78-21419a04c8dd service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:38.812756 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a5b95-3806-4648-8a78-21419a04c8dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:38.812900 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a5b95-3806-4648-8a78-21419a04c8dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:38.828679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a5b95-3806-4648-8a78-21419a04c8dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:38.830711 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a5b95-3806-4648-8a78-21419a04c8dd service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:38.854652 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1150/2282] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:36:39.312525 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ac41e-f83b-4118-9c1b-eeef17e402a5 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:39.312973 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ac41e-f83b-4118-9c1b-eeef17e402a5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:39.312973 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ac41e-f83b-4118-9c1b-eeef17e402a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:39.341363 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-6f0ac41e-f83b-4118-9c1b-eeef17e402a5 None None] MFA Rules not processed for user `765c911957bd4ca68dd520fc4fbba1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:39.344246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6f0ac41e-f83b-4118-9c1b-eeef17e402a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:39.531545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1133/2283] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:39.571083 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a5b95-3806-4648-8a78-21419a04c8dd service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:39.790124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a5b95-3806-4648-8a78-21419a04c8dd service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:39.801372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14d5e2-327d-4ac3-9b8e-2e0e71b14947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:39.803747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14d5e2-327d-4ac3-9b8e-2e0e71b14947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:39.803949 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14d5e2-327d-4ac3-9b8e-2e0e71b14947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:39.804235 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14d5e2-327d-4ac3-9b8e-2e0e71b14947 admin admin] PATH_INFO: `/v3/users/c66283b368a049939a52d554b489dccf` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:39.807492 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14d5e2-327d-4ac3-9b8e-2e0e71b14947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c66283b368a049939a52d554b489dccf)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:39.809560 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14d5e2-327d-4ac3-9b8e-2e0e71b14947 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:39.928747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6b14d5e2-327d-4ac3-9b8e-2e0e71b14947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c66283b368a049939a52d554b489dccf'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:39.941429 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6b14d5e2-327d-4ac3-9b8e-2e0e71b14947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c66283b368a049939a52d554b489dccf'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:39.953791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6b14d5e2-327d-4ac3-9b8e-2e0e71b14947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c66283b368a049939a52d554b489dccf'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:39.954815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6b14d5e2-327d-4ac3-9b8e-2e0e71b14947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c66283b368a049939a52d554b489dccf'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:39.955668 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6b14d5e2-327d-4ac3-9b8e-2e0e71b14947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c66283b368a049939a52d554b489dccf'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:39.984421 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1151/2284] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:39 2022] DELETE /identity/v3/users/c66283b368a049939a52d554b489dccf => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:39.986708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48948fc9-c558-4c24-975a-66aebd2e22df None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:39.987033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48948fc9-c558-4c24-975a-66aebd2e22df None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:39.987347 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48948fc9-c558-4c24-975a-66aebd2e22df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:40.016075 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-48948fc9-c558-4c24-975a-66aebd2e22df None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:40.021901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-48948fc9-c558-4c24-975a-66aebd2e22df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:40.055649 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:40.178468 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1134/2285] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:40.238239 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14d5e2-327d-4ac3-9b8e-2e0e71b14947 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:40.240305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6b14d5e2-327d-4ac3-9b8e-2e0e71b14947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:40.320148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:40.343725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14d5e2-327d-4ac3-9b8e-2e0e71b14947 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:40.358295 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6c340-9cdb-4b51-b2cb-87094fc4317a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:40.360045 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6c340-9cdb-4b51-b2cb-87094fc4317a service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:40.361478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6c340-9cdb-4b51-b2cb-87094fc4317a service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:40.361784 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6c340-9cdb-4b51-b2cb-87094fc4317a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:40.387692 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6c340-9cdb-4b51-b2cb-87094fc4317a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:40.389900 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6c340-9cdb-4b51-b2cb-87094fc4317a service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:40.412470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1152/2286] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:41.228549 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ece87-fc85-4529-a577-2eb247a7d52a None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:41.229124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ece87-fc85-4529-a577-2eb247a7d52a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:41.229124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ece87-fc85-4529-a577-2eb247a7d52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:41.284995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d93ece87-fc85-4529-a577-2eb247a7d52a None None] MFA Rules not processed for user `a1f5c428bab24681b4b91bc04f200f7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:41.286244 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d93ece87-fc85-4529-a577-2eb247a7d52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:41.438016 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6c340-9cdb-4b51-b2cb-87094fc4317a service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:41.440526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4ce6c340-9cdb-4b51-b2cb-87094fc4317a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:41.465750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6c340-9cdb-4b51-b2cb-87094fc4317a service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:41.477261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc951cf-51ab-4911-9258-3686640ba5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:41.478424 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc951cf-51ab-4911-9258-3686640ba5d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:41.479484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc951cf-51ab-4911-9258-3686640ba5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:41.481495 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc951cf-51ab-4911-9258-3686640ba5d3 admin admin] PATH_INFO: `/v3/projects/7b1e02409208460893f61eb01b2cabb4` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:41.483889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc951cf-51ab-4911-9258-3686640ba5d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b1e02409208460893f61eb01b2cabb4)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:41.488739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc951cf-51ab-4911-9258-3686640ba5d3 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:41.527568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1135/2287] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:41.549822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ece87-fc85-4529-a577-2eb247a7d52a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:41.553863 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d93ece87-fc85-4529-a577-2eb247a7d52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:41.617502 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-efc951cf-51ab-4911-9258-3686640ba5d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b1e02409208460893f61eb01b2cabb4', 'request_id': 'req-efc951cf-51ab-4911-9258-3686640ba5d3'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:41.653635 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-efc951cf-51ab-4911-9258-3686640ba5d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b1e02409208460893f61eb01b2cabb4', 'request_id': 'req-efc951cf-51ab-4911-9258-3686640ba5d3'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:41.671664 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-efc951cf-51ab-4911-9258-3686640ba5d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b1e02409208460893f61eb01b2cabb4', 'request_id': 'req-efc951cf-51ab-4911-9258-3686640ba5d3'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:41.700155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-efc951cf-51ab-4911-9258-3686640ba5d3 admin admin] The token cache is being invalidate because project 7b1e02409208460893f61eb01b2cabb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:41.700383 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-efc951cf-51ab-4911-9258-3686640ba5d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:41.702414 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-efc951cf-51ab-4911-9258-3686640ba5d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:41.713298 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1153/2288] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:41 2022] DELETE /identity/v3/projects/7b1e02409208460893f61eb01b2cabb4 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:36:41.718043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc951cf-51ab-4911-9258-3686640ba5d3 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:41.725252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-efc951cf-51ab-4911-9258-3686640ba5d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:41.817925 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ece87-fc85-4529-a577-2eb247a7d52a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:41.898875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:41.904495 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc951cf-51ab-4911-9258-3686640ba5d3 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:41.917305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f21b2-a666-46ca-a95e-d205fa4832a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:41.918277 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f21b2-a666-46ca-a95e-d205fa4832a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:41.918637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f21b2-a666-46ca-a95e-d205fa4832a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:41.918819 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f21b2-a666-46ca-a95e-d205fa4832a4 admin admin] PATH_INFO: `/v3/users/cb90f2a4013c4d78a49615189cc8ff56` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:41.921392 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f21b2-a666-46ca-a95e-d205fa4832a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb90f2a4013c4d78a49615189cc8ff56)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:41.923641 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f21b2-a666-46ca-a95e-d205fa4832a4 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:41.963843 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9633603-ba53-4327-84cb-67c378644b1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:41.966922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9633603-ba53-4327-84cb-67c378644b1c service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:41.966922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9633603-ba53-4327-84cb-67c378644b1c service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:41.966922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9633603-ba53-4327-84cb-67c378644b1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:41.987727 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b9633603-ba53-4327-84cb-67c378644b1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:42.107742 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-587f21b2-a666-46ca-a95e-d205fa4832a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb90f2a4013c4d78a49615189cc8ff56'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:42.130032 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-587f21b2-a666-46ca-a95e-d205fa4832a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb90f2a4013c4d78a49615189cc8ff56'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:42.152179 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-587f21b2-a666-46ca-a95e-d205fa4832a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb90f2a4013c4d78a49615189cc8ff56'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:42.153327 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-587f21b2-a666-46ca-a95e-d205fa4832a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb90f2a4013c4d78a49615189cc8ff56'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:42.154374 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9633603-ba53-4327-84cb-67c378644b1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:42.155471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-587f21b2-a666-46ca-a95e-d205fa4832a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb90f2a4013c4d78a49615189cc8ff56'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:42.157502 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:42.165648 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9633603-ba53-4327-84cb-67c378644b1c service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:42.194434 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1136/2289] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:42.212004 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1154/2290] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:41 2022] DELETE /identity/v3/users/cb90f2a4013c4d78a49615189cc8ff56 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:42.640526 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb1406-4652-4eef-b892-eb4f503258a4 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:42.641023 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb1406-4652-4eef-b892-eb4f503258a4 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:42.641152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb1406-4652-4eef-b892-eb4f503258a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:42.670282 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8cdb1406-4652-4eef-b892-eb4f503258a4 None None] MFA Rules not processed for user `16c6975c73954487a1f15649693b1d8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:42.671642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8cdb1406-4652-4eef-b892-eb4f503258a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:42.756633 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1137/2291] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:42.929829 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f21b2-a666-46ca-a95e-d205fa4832a4 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:42.931595 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-587f21b2-a666-46ca-a95e-d205fa4832a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:43.096717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f21b2-a666-46ca-a95e-d205fa4832a4 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:43.102595 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d71cd3-3de3-4377-a73b-9d88a9dd7617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:43.104507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d71cd3-3de3-4377-a73b-9d88a9dd7617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:43.105521 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d71cd3-3de3-4377-a73b-9d88a9dd7617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:43.106516 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d71cd3-3de3-4377-a73b-9d88a9dd7617 admin admin] PATH_INFO: `/v3/projects/cee1ea0a153340ddbbd3a8bbef497162` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:43.108195 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d71cd3-3de3-4377-a73b-9d88a9dd7617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee1ea0a153340ddbbd3a8bbef497162)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:43.110538 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d71cd3-3de3-4377-a73b-9d88a9dd7617 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:43.232286 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-a7d71cd3-3de3-4377-a73b-9d88a9dd7617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee1ea0a153340ddbbd3a8bbef497162', 'request_id': 'req-a7d71cd3-3de3-4377-a73b-9d88a9dd7617'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:43.245528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-a7d71cd3-3de3-4377-a73b-9d88a9dd7617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee1ea0a153340ddbbd3a8bbef497162', 'request_id': 'req-a7d71cd3-3de3-4377-a73b-9d88a9dd7617'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:43.258577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-a7d71cd3-3de3-4377-a73b-9d88a9dd7617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee1ea0a153340ddbbd3a8bbef497162', 'request_id': 'req-a7d71cd3-3de3-4377-a73b-9d88a9dd7617'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:43.278686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-a7d71cd3-3de3-4377-a73b-9d88a9dd7617 admin admin] The token cache is being invalidate because project cee1ea0a153340ddbbd3a8bbef497162 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:43.279048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-a7d71cd3-3de3-4377-a73b-9d88a9dd7617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:43.279705 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-a7d71cd3-3de3-4377-a73b-9d88a9dd7617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:43.285153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1155/2292] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:42 2022] DELETE /identity/v3/projects/cee1ea0a153340ddbbd3a8bbef497162 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:43.286870 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1f47b-162c-470e-91bb-d08e230315c6 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:43.287207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1f47b-162c-470e-91bb-d08e230315c6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:43.287488 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1f47b-162c-470e-91bb-d08e230315c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:43.313399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-99c1f47b-162c-470e-91bb-d08e230315c6 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:43.315671 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-99c1f47b-162c-470e-91bb-d08e230315c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:43.391870 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1138/2293] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:43.400878 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9e563-e74b-4af7-90fc-4ec163b27ecc None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:43.401121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9e563-e74b-4af7-90fc-4ec163b27ecc None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:43.401380 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9e563-e74b-4af7-90fc-4ec163b27ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:43.422833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-79c9e563-e74b-4af7-90fc-4ec163b27ecc None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:43.424365 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-79c9e563-e74b-4af7-90fc-4ec163b27ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:43.434906 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1156/2294] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:43.439614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1f47b-162c-470e-91bb-d08e230315c6 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:43.454076 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1f47b-162c-470e-91bb-d08e230315c6 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:43.462347 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca704ef-7267-4274-87f5-16eb2ae1367a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:43.463985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca704ef-7267-4274-87f5-16eb2ae1367a admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:43.464803 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca704ef-7267-4274-87f5-16eb2ae1367a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:43.465265 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca704ef-7267-4274-87f5-16eb2ae1367a admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:43.466120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca704ef-7267-4274-87f5-16eb2ae1367a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:36:43.466510 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca704ef-7267-4274-87f5-16eb2ae1367a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:43.468628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca704ef-7267-4274-87f5-16eb2ae1367a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:43.493743 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1139/2295] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:36:43.497046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9e563-e74b-4af7-90fc-4ec163b27ecc None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:43.507362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9e563-e74b-4af7-90fc-4ec163b27ecc None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:43.514606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821ee49-a160-4e4a-89d2-94e41261f0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:43.515456 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821ee49-a160-4e4a-89d2-94e41261f0c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:43.515816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821ee49-a160-4e4a-89d2-94e41261f0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:43.516193 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821ee49-a160-4e4a-89d2-94e41261f0c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:43.517743 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821ee49-a160-4e4a-89d2-94e41261f0c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:43.520682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821ee49-a160-4e4a-89d2-94e41261f0c7 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:43.578485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1157/2296] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:43 2022] POST /identity/v3/projects => generated 379 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:43.580382 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca704ef-7267-4274-87f5-16eb2ae1367a admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:43.596596 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca704ef-7267-4274-87f5-16eb2ae1367a admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:43.614215 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:43.694357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cea13-0c0d-467f-afcb-6aa8d13bd580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:43.695133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cea13-0c0d-467f-afcb-6aa8d13bd580 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:43.695421 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cea13-0c0d-467f-afcb-6aa8d13bd580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:43.695696 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cea13-0c0d-467f-afcb-6aa8d13bd580 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:43.696399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cea13-0c0d-467f-afcb-6aa8d13bd580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:43.698350 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cea13-0c0d-467f-afcb-6aa8d13bd580 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:43.727953 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1140/2297] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:43 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:43.732214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821ee49-a160-4e4a-89d2-94e41261f0c7 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:43.745275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821ee49-a160-4e4a-89d2-94e41261f0c7 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:43.753701 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a14c63-2fd5-495f-843b-9e84d6c92b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:43.754319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a14c63-2fd5-495f-843b-9e84d6c92b9b admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:43.754607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a14c63-2fd5-495f-843b-9e84d6c92b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:43.754883 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a14c63-2fd5-495f-843b-9e84d6c92b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:43.755672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a14c63-2fd5-495f-843b-9e84d6c92b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:43.761687 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a14c63-2fd5-495f-843b-9e84d6c92b9b admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:43.785399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1158/2298] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:43.790411 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cea13-0c0d-467f-afcb-6aa8d13bd580 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:43.803762 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cea13-0c0d-467f-afcb-6aa8d13bd580 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:43.809163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f1d2e-8444-4e2a-bee9-231143eae47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:43.810330 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f1d2e-8444-4e2a-bee9-231143eae47c admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:43.810624 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f1d2e-8444-4e2a-bee9-231143eae47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:43.810909 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f1d2e-8444-4e2a-bee9-231143eae47c admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:43.811769 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f1d2e-8444-4e2a-bee9-231143eae47c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:43.813643 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f1d2e-8444-4e2a-bee9-231143eae47c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:43.830028 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1141/2299] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:43.831179 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a14c63-2fd5-495f-843b-9e84d6c92b9b admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:43.852065 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a14c63-2fd5-495f-843b-9e84d6c92b9b admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:43.858209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61768f0d-e825-4d5e-9ecf-85f7a78f4ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:43.859975 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61768f0d-e825-4d5e-9ecf-85f7a78f4ff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:43.860476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61768f0d-e825-4d5e-9ecf-85f7a78f4ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:43.860968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61768f0d-e825-4d5e-9ecf-85f7a78f4ff9 admin admin] PATH_INFO: `/v3/projects/818090d8ac2141adaff61780ac5e13ca/users/2c834c3a8a154c80b420b82f17e99379/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:43.895612 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61768f0d-e825-4d5e-9ecf-85f7a78f4ff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=818090d8ac2141adaff61780ac5e13ca, user_id=2c834c3a8a154c80b420b82f17e99379, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:43.898569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61768f0d-e825-4d5e-9ecf-85f7a78f4ff9 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:43.924785 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1159/2300] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:43 2022] PUT /identity/v3/projects/818090d8ac2141adaff61780ac5e13ca/users/2c834c3a8a154c80b420b82f17e99379/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:36:43.936414 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ebf6d-2060-4d03-973b-65d2db339338 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:43.936753 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ebf6d-2060-4d03-973b-65d2db339338 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:43.937058 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ebf6d-2060-4d03-973b-65d2db339338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:44.000987 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-026ebf6d-2060-4d03-973b-65d2db339338 None None] MFA Rules not processed for user `2c834c3a8a154c80b420b82f17e99379`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:44.002832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-026ebf6d-2060-4d03-973b-65d2db339338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:44.164407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1142/2301] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:44.168764 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61768f0d-e825-4d5e-9ecf-85f7a78f4ff9 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:44.186174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:44.269851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61768f0d-e825-4d5e-9ecf-85f7a78f4ff9 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:44.274845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b6e85-c0d6-432d-aca3-a8a9248c0a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:44.275611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b6e85-c0d6-432d-aca3-a8a9248c0a2b admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:44.275895 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b6e85-c0d6-432d-aca3-a8a9248c0a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:44.276171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b6e85-c0d6-432d-aca3-a8a9248c0a2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:44.276872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b6e85-c0d6-432d-aca3-a8a9248c0a2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:44.278789 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b6e85-c0d6-432d-aca3-a8a9248c0a2b admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:44.326341 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1160/2302] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:44 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:44.327800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ebf6d-2060-4d03-973b-65d2db339338 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:44.354503 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ebf6d-2060-4d03-973b-65d2db339338 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:44.447161 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301afc6-cadb-4989-b9fd-da0d55d32144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:44.448221 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301afc6-cadb-4989-b9fd-da0d55d32144 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:44.448777 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301afc6-cadb-4989-b9fd-da0d55d32144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:44.449275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301afc6-cadb-4989-b9fd-da0d55d32144 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:44.450482 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301afc6-cadb-4989-b9fd-da0d55d32144 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:44.454042 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301afc6-cadb-4989-b9fd-da0d55d32144 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:44.489016 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1143/2303] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:44 2022] POST /identity/v3/users => generated 296 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:44.493119 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b6e85-c0d6-432d-aca3-a8a9248c0a2b admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:44.506022 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b6e85-c0d6-432d-aca3-a8a9248c0a2b admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:44.513258 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6150a2-df98-421a-a7b9-c364e6e754e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:44.514383 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6150a2-df98-421a-a7b9-c364e6e754e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:44.514792 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6150a2-df98-421a-a7b9-c364e6e754e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:44.515309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6150a2-df98-421a-a7b9-c364e6e754e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:44.517132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6150a2-df98-421a-a7b9-c364e6e754e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:44.519694 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6150a2-df98-421a-a7b9-c364e6e754e5 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:44.538330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1161/2304] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:44.547735 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301afc6-cadb-4989-b9fd-da0d55d32144 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:44.566428 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301afc6-cadb-4989-b9fd-da0d55d32144 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:44.572102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09deb391-1317-40cc-ba9b-e4a24de7ec7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:44.573478 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09deb391-1317-40cc-ba9b-e4a24de7ec7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:44.574167 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09deb391-1317-40cc-ba9b-e4a24de7ec7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:44.574894 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09deb391-1317-40cc-ba9b-e4a24de7ec7c admin admin] PATH_INFO: `/v3/projects/483cd87d16b14806ab6d56d739c3991f/users/2ea4de2b07f144178b89e657c079c025/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:44.587134 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:44.614835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09deb391-1317-40cc-ba9b-e4a24de7ec7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=483cd87d16b14806ab6d56d739c3991f, user_id=2ea4de2b07f144178b89e657c079c025, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:44.618680 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09deb391-1317-40cc-ba9b-e4a24de7ec7c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:44.673827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1144/2305] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:44 2022] PUT /identity/v3/projects/483cd87d16b14806ab6d56d739c3991f/users/2ea4de2b07f144178b89e657c079c025/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:44.677762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6150a2-df98-421a-a7b9-c364e6e754e5 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:44.690049 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6150a2-df98-421a-a7b9-c364e6e754e5 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:44.765353 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa5363-4e22-4e13-8af2-30759dfbffa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:44.766176 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa5363-4e22-4e13-8af2-30759dfbffa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:44.766528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa5363-4e22-4e13-8af2-30759dfbffa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:44.766856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa5363-4e22-4e13-8af2-30759dfbffa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:44.768010 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa5363-4e22-4e13-8af2-30759dfbffa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:44.770000 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa5363-4e22-4e13-8af2-30759dfbffa8 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:44.784909 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1162/2306] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:44.786146 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09deb391-1317-40cc-ba9b-e4a24de7ec7c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:44.797745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09deb391-1317-40cc-ba9b-e4a24de7ec7c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:44.803434 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec074e-6e7d-4c5b-96d3-fefce3910705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:44.804325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec074e-6e7d-4c5b-96d3-fefce3910705 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:44.804608 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec074e-6e7d-4c5b-96d3-fefce3910705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:44.804878 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec074e-6e7d-4c5b-96d3-fefce3910705 admin admin] PATH_INFO: `/v3/projects/483cd87d16b14806ab6d56d739c3991f/users/2ea4de2b07f144178b89e657c079c025/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:44.808385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec074e-6e7d-4c5b-96d3-fefce3910705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=483cd87d16b14806ab6d56d739c3991f, user_id=2ea4de2b07f144178b89e657c079c025, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:44.809524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec074e-6e7d-4c5b-96d3-fefce3910705 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:44.829734 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1145/2307] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:44 2022] PUT /identity/v3/projects/483cd87d16b14806ab6d56d739c3991f/users/2ea4de2b07f144178b89e657c079c025/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:44.836226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47b0f0-e47b-4c13-9f0d-71ad3de76474 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:44.836578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47b0f0-e47b-4c13-9f0d-71ad3de76474 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:44.836860 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47b0f0-e47b-4c13-9f0d-71ad3de76474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:44.868078 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-6c47b0f0-e47b-4c13-9f0d-71ad3de76474 None None] MFA Rules not processed for user `2ea4de2b07f144178b89e657c079c025`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:44.869714 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6c47b0f0-e47b-4c13-9f0d-71ad3de76474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:44.961296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1163/2308] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:44.976105 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec074e-6e7d-4c5b-96d3-fefce3910705 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:44.977763 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-aeec074e-6e7d-4c5b-96d3-fefce3910705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:45.057959 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec074e-6e7d-4c5b-96d3-fefce3910705 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:45.060834 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5ec3f-2ff6-4f4e-9f2c-57306f19edce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:45.061661 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5ec3f-2ff6-4f4e-9f2c-57306f19edce service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:45.061894 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5ec3f-2ff6-4f4e-9f2c-57306f19edce service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:45.062172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5ec3f-2ff6-4f4e-9f2c-57306f19edce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:45.072556 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5ec3f-2ff6-4f4e-9f2c-57306f19edce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:45.073993 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5ec3f-2ff6-4f4e-9f2c-57306f19edce service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:45.102545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1146/2309] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:45.455512 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47b0f0-e47b-4c13-9f0d-71ad3de76474 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:45.460016 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:45.467435 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47b0f0-e47b-4c13-9f0d-71ad3de76474 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:45.473328 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61c607-14a8-483d-a138-bfd94a4737ec tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ea4de2b07f144178b89e657c079c025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '483cd87d16b14806ab6d56d739c3991f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:45.474002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61c607-14a8-483d-a138-bfd94a4737ec tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:45.474294 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61c607-14a8-483d-a138-bfd94a4737ec tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:45.474571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61c607-14a8-483d-a138-bfd94a4737ec tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:45.475355 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61c607-14a8-483d-a138-bfd94a4737ec tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:36:45.475663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61c607-14a8-483d-a138-bfd94a4737ec tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:45.477465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61c607-14a8-483d-a138-bfd94a4737ec tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:45.489349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1164/2310] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:36:45.493526 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5ec3f-2ff6-4f4e-9f2c-57306f19edce service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:45.503547 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5ec3f-2ff6-4f4e-9f2c-57306f19edce service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:45.508726 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf2289-ebde-4c88-bb03-8f9b072f1167 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ea4de2b07f144178b89e657c079c025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '483cd87d16b14806ab6d56d739c3991f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:45.509576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf2289-ebde-4c88-bb03-8f9b072f1167 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:45.509919 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf2289-ebde-4c88-bb03-8f9b072f1167 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:45.510270 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf2289-ebde-4c88-bb03-8f9b072f1167 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:45.511175 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf2289-ebde-4c88-bb03-8f9b072f1167 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:45.513463 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf2289-ebde-4c88-bb03-8f9b072f1167 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:45.548485 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1147/2311] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:45 2022] POST /identity/v3/projects => generated 354 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:45.549789 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61c607-14a8-483d-a138-bfd94a4737ec tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:45.560925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61c607-14a8-483d-a138-bfd94a4737ec tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:45.651593 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48b146-215e-47c3-9c0e-8274929f1ca6 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ea4de2b07f144178b89e657c079c025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '483cd87d16b14806ab6d56d739c3991f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:45.652419 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48b146-215e-47c3-9c0e-8274929f1ca6 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:45.652815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48b146-215e-47c3-9c0e-8274929f1ca6 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:45.653197 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48b146-215e-47c3-9c0e-8274929f1ca6 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:45.654066 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48b146-215e-47c3-9c0e-8274929f1ca6 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:36:45.654486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48b146-215e-47c3-9c0e-8274929f1ca6 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:45.656458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48b146-215e-47c3-9c0e-8274929f1ca6 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:45.677425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1165/2312] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:36:45.719293 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf2289-ebde-4c88-bb03-8f9b072f1167 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:45.742237 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf2289-ebde-4c88-bb03-8f9b072f1167 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:45.748619 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d10421-eed5-49df-aa2a-0c2c675f0a57 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ea4de2b07f144178b89e657c079c025', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '483cd87d16b14806ab6d56d739c3991f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:45.749572 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d10421-eed5-49df-aa2a-0c2c675f0a57 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:45.749859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d10421-eed5-49df-aa2a-0c2c675f0a57 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:45.750131 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d10421-eed5-49df-aa2a-0c2c675f0a57 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] PATH_INFO: `/v3/projects/1d4619778fef4b458a605ac6e089a4bc` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:45.752367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d10421-eed5-49df-aa2a-0c2c675f0a57 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1d4619778fef4b458a605ac6e089a4bc)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:45.754749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d10421-eed5-49df-aa2a-0c2c675f0a57 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:45.889753 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-42d10421-eed5-49df-aa2a-0c2c675f0a57 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d4619778fef4b458a605ac6e089a4bc', 'request_id': 'req-42d10421-eed5-49df-aa2a-0c2c675f0a57'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:45.903821 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-42d10421-eed5-49df-aa2a-0c2c675f0a57 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d4619778fef4b458a605ac6e089a4bc', 'request_id': 'req-42d10421-eed5-49df-aa2a-0c2c675f0a57'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:45.917228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-42d10421-eed5-49df-aa2a-0c2c675f0a57 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d4619778fef4b458a605ac6e089a4bc', 'request_id': 'req-42d10421-eed5-49df-aa2a-0c2c675f0a57'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:45.946822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-42d10421-eed5-49df-aa2a-0c2c675f0a57 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] The token cache is being invalidate because project 1d4619778fef4b458a605ac6e089a4bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:45.948079 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-42d10421-eed5-49df-aa2a-0c2c675f0a57 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:45.949266 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-42d10421-eed5-49df-aa2a-0c2c675f0a57 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:45.952652 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1148/2313] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:45 2022] DELETE /identity/v3/projects/1d4619778fef4b458a605ac6e089a4bc => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:46.417109 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48b146-215e-47c3-9c0e-8274929f1ca6 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:46.419319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-da48b146-215e-47c3-9c0e-8274929f1ca6 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:46.517509 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48b146-215e-47c3-9c0e-8274929f1ca6 tempest-VolumeQuotaClassesTest-993973946 tempest-VolumeQuotaClassesTest-993973946-project-admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:46.609630 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58756e24-4f45-4e14-a157-63e1fd4a9d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:46.612210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58756e24-4f45-4e14-a157-63e1fd4a9d3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:46.612582 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58756e24-4f45-4e14-a157-63e1fd4a9d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:46.613646 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58756e24-4f45-4e14-a157-63e1fd4a9d3b admin admin] PATH_INFO: `/v3/users/2c834c3a8a154c80b420b82f17e99379` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:46.618007 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58756e24-4f45-4e14-a157-63e1fd4a9d3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c834c3a8a154c80b420b82f17e99379)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:46.623866 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58756e24-4f45-4e14-a157-63e1fd4a9d3b admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:46.718630 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:46.781023 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-58756e24-4f45-4e14-a157-63e1fd4a9d3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c834c3a8a154c80b420b82f17e99379'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:46.793850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-58756e24-4f45-4e14-a157-63e1fd4a9d3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c834c3a8a154c80b420b82f17e99379'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:46.804657 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-58756e24-4f45-4e14-a157-63e1fd4a9d3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c834c3a8a154c80b420b82f17e99379'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:46.805417 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-58756e24-4f45-4e14-a157-63e1fd4a9d3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c834c3a8a154c80b420b82f17e99379'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:46.806059 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-58756e24-4f45-4e14-a157-63e1fd4a9d3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c834c3a8a154c80b420b82f17e99379'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:46.843256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1166/2314] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:46 2022] DELETE /identity/v3/users/2c834c3a8a154c80b420b82f17e99379 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:46.851120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afd15a-0df6-482c-8fdd-bec880690685 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:46.851956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afd15a-0df6-482c-8fdd-bec880690685 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:46.852906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afd15a-0df6-482c-8fdd-bec880690685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:46.904450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-20afd15a-0df6-482c-8fdd-bec880690685 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:46.906436 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-20afd15a-0df6-482c-8fdd-bec880690685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:46.959042 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:46.990641 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1149/2315] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:47.001628 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58756e24-4f45-4e14-a157-63e1fd4a9d3b admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:47.003373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-58756e24-4f45-4e14-a157-63e1fd4a9d3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:47.075247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58756e24-4f45-4e14-a157-63e1fd4a9d3b admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:47.079038 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d0ff4-8233-42cc-b466-7c7ce83d1a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:47.080263 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d0ff4-8233-42cc-b466-7c7ce83d1a42 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:47.080652 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d0ff4-8233-42cc-b466-7c7ce83d1a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:47.080827 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d0ff4-8233-42cc-b466-7c7ce83d1a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:47.090875 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d0ff4-8233-42cc-b466-7c7ce83d1a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:47.092359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d0ff4-8233-42cc-b466-7c7ce83d1a42 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:47.103534 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1167/2316] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:47.570880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afd15a-0df6-482c-8fdd-bec880690685 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:47.572780 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-20afd15a-0df6-482c-8fdd-bec880690685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:47.586654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afd15a-0df6-482c-8fdd-bec880690685 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:47.592237 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd009e9-762e-4de1-8474-897932465415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:47.593438 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd009e9-762e-4de1-8474-897932465415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:47.593872 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd009e9-762e-4de1-8474-897932465415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:47.594469 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd009e9-762e-4de1-8474-897932465415 admin admin] PATH_INFO: `/v3/projects/818090d8ac2141adaff61780ac5e13ca` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:47.596108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd009e9-762e-4de1-8474-897932465415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=818090d8ac2141adaff61780ac5e13ca)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:47.598567 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd009e9-762e-4de1-8474-897932465415 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:47.707816 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7dd009e9-762e-4de1-8474-897932465415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '818090d8ac2141adaff61780ac5e13ca', 'request_id': 'req-7dd009e9-762e-4de1-8474-897932465415'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:47.717817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7dd009e9-762e-4de1-8474-897932465415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '818090d8ac2141adaff61780ac5e13ca', 'request_id': 'req-7dd009e9-762e-4de1-8474-897932465415'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:47.727530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7dd009e9-762e-4de1-8474-897932465415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '818090d8ac2141adaff61780ac5e13ca', 'request_id': 'req-7dd009e9-762e-4de1-8474-897932465415'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:47.745065 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7dd009e9-762e-4de1-8474-897932465415 admin admin] The token cache is being invalidate because project 818090d8ac2141adaff61780ac5e13ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:47.745484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7dd009e9-762e-4de1-8474-897932465415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:47.746461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7dd009e9-762e-4de1-8474-897932465415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:47.751649 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1150/2317] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:47 2022] DELETE /identity/v3/projects/818090d8ac2141adaff61780ac5e13ca => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:47.753780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d0ff4-8233-42cc-b466-7c7ce83d1a42 service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:47.781699 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d0ff4-8233-42cc-b466-7c7ce83d1a42 service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:47.978541 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d25632-579a-4c43-ba95-5aa60475f7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:47.980099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d25632-579a-4c43-ba95-5aa60475f7d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:47.980517 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d25632-579a-4c43-ba95-5aa60475f7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:47.981097 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d25632-579a-4c43-ba95-5aa60475f7d4 admin admin] PATH_INFO: `/v3/users/2ea4de2b07f144178b89e657c079c025` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:47.983109 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d25632-579a-4c43-ba95-5aa60475f7d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ea4de2b07f144178b89e657c079c025)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:47.987254 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d25632-579a-4c43-ba95-5aa60475f7d4 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:48.056278 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-29d25632-579a-4c43-ba95-5aa60475f7d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ea4de2b07f144178b89e657c079c025'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:48.068487 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-29d25632-579a-4c43-ba95-5aa60475f7d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ea4de2b07f144178b89e657c079c025'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:48.083353 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:48.084771 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-29d25632-579a-4c43-ba95-5aa60475f7d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ea4de2b07f144178b89e657c079c025'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:48.086091 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-29d25632-579a-4c43-ba95-5aa60475f7d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ea4de2b07f144178b89e657c079c025'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:48.087537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-29d25632-579a-4c43-ba95-5aa60475f7d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ea4de2b07f144178b89e657c079c025'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:48.120605 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1168/2318] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:47 2022] DELETE /identity/v3/users/2ea4de2b07f144178b89e657c079c025 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:48.565230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd009e9-762e-4de1-8474-897932465415 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:48.567008 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7dd009e9-762e-4de1-8474-897932465415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:48.586684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd009e9-762e-4de1-8474-897932465415 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:48.675981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff57e9c-cdd8-4344-bce2-75c0f4b4ed94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:48.676984 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff57e9c-cdd8-4344-bce2-75c0f4b4ed94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:48.677491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff57e9c-cdd8-4344-bce2-75c0f4b4ed94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:48.677615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff57e9c-cdd8-4344-bce2-75c0f4b4ed94 admin admin] PATH_INFO: `/v3/projects/483cd87d16b14806ab6d56d739c3991f` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:48.681805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff57e9c-cdd8-4344-bce2-75c0f4b4ed94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=483cd87d16b14806ab6d56d739c3991f)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:48.683880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff57e9c-cdd8-4344-bce2-75c0f4b4ed94 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:48.753719 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aff57e9c-cdd8-4344-bce2-75c0f4b4ed94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '483cd87d16b14806ab6d56d739c3991f', 'request_id': 'req-aff57e9c-cdd8-4344-bce2-75c0f4b4ed94'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:48.761953 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aff57e9c-cdd8-4344-bce2-75c0f4b4ed94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '483cd87d16b14806ab6d56d739c3991f', 'request_id': 'req-aff57e9c-cdd8-4344-bce2-75c0f4b4ed94'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:48.772644 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aff57e9c-cdd8-4344-bce2-75c0f4b4ed94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '483cd87d16b14806ab6d56d739c3991f', 'request_id': 'req-aff57e9c-cdd8-4344-bce2-75c0f4b4ed94'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:48.778073 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:48.798745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aff57e9c-cdd8-4344-bce2-75c0f4b4ed94 admin admin] The token cache is being invalidate because project 483cd87d16b14806ab6d56d739c3991f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:36:48.799094 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aff57e9c-cdd8-4344-bce2-75c0f4b4ed94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:48.799868 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aff57e9c-cdd8-4344-bce2-75c0f4b4ed94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:48.802084 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1151/2319] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:36:48 2022] DELETE /identity/v3/projects/483cd87d16b14806ab6d56d739c3991f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:48.807885 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915b93f-cbef-41e1-93a0-1e30cbc06a5a None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:48.808504 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915b93f-cbef-41e1-93a0-1e30cbc06a5a None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:48.808844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915b93f-cbef-41e1-93a0-1e30cbc06a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:48.839946 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-3915b93f-cbef-41e1-93a0-1e30cbc06a5a None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:48.842042 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3915b93f-cbef-41e1-93a0-1e30cbc06a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:48.938364 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1169/2320] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:48.947539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f320a4c-98b9-494f-adb2-c919fd8d4bff None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:48.948023 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f320a4c-98b9-494f-adb2-c919fd8d4bff None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:48.948278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f320a4c-98b9-494f-adb2-c919fd8d4bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:48.972010 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8f320a4c-98b9-494f-adb2-c919fd8d4bff None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:48.973518 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8f320a4c-98b9-494f-adb2-c919fd8d4bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:48.986721 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1152/2321] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:36:48.988099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915b93f-cbef-41e1-93a0-1e30cbc06a5a None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:49.004610 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915b93f-cbef-41e1-93a0-1e30cbc06a5a None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:49.011503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f085a4-5359-4493-ab90-46d40af94035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:49.012429 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f085a4-5359-4493-ab90-46d40af94035 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:49.012843 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f085a4-5359-4493-ab90-46d40af94035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:49.013255 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f085a4-5359-4493-ab90-46d40af94035 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:49.014185 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f085a4-5359-4493-ab90-46d40af94035 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:36:49.014801 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f085a4-5359-4493-ab90-46d40af94035 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:49.017208 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f085a4-5359-4493-ab90-46d40af94035 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:49.038612 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1170/2322] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:36:49.040079 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f320a4c-98b9-494f-adb2-c919fd8d4bff None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:49.052538 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f320a4c-98b9-494f-adb2-c919fd8d4bff None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:49.058077 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a12f4c-59ad-4c1c-b67e-34da2c110816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:49.058921 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a12f4c-59ad-4c1c-b67e-34da2c110816 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:49.059356 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a12f4c-59ad-4c1c-b67e-34da2c110816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:49.059723 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a12f4c-59ad-4c1c-b67e-34da2c110816 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:49.060692 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a12f4c-59ad-4c1c-b67e-34da2c110816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:49.063084 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a12f4c-59ad-4c1c-b67e-34da2c110816 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:49.091207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1153/2323] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:49 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:36:49.095860 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f085a4-5359-4493-ab90-46d40af94035 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:49.109334 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f085a4-5359-4493-ab90-46d40af94035 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:49.180611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66307176-6c8b-4cd1-b90e-9ae68a71c581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:49.181393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66307176-6c8b-4cd1-b90e-9ae68a71c581 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:49.181729 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66307176-6c8b-4cd1-b90e-9ae68a71c581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:49.182057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66307176-6c8b-4cd1-b90e-9ae68a71c581 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:49.182911 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66307176-6c8b-4cd1-b90e-9ae68a71c581 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:49.185192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66307176-6c8b-4cd1-b90e-9ae68a71c581 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:49.210310 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1171/2324] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:49 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:49.211925 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a12f4c-59ad-4c1c-b67e-34da2c110816 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:49.227284 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a12f4c-59ad-4c1c-b67e-34da2c110816 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:49.232825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd6ea6-5e18-42fa-ace4-da56d8f6babb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:49.233590 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd6ea6-5e18-42fa-ace4-da56d8f6babb admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:49.233944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd6ea6-5e18-42fa-ace4-da56d8f6babb admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:49.234299 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd6ea6-5e18-42fa-ace4-da56d8f6babb admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:49.235176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd6ea6-5e18-42fa-ace4-da56d8f6babb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:49.237389 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd6ea6-5e18-42fa-ace4-da56d8f6babb admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:49.248910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1154/2325] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:49.253417 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66307176-6c8b-4cd1-b90e-9ae68a71c581 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:49.264173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66307176-6c8b-4cd1-b90e-9ae68a71c581 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:49.269966 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b730e-0d78-4612-9403-40882e45d3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:49.270640 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b730e-0d78-4612-9403-40882e45d3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:49.270926 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b730e-0d78-4612-9403-40882e45d3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:49.271490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b730e-0d78-4612-9403-40882e45d3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:49.272223 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b730e-0d78-4612-9403-40882e45d3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:49.274192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b730e-0d78-4612-9403-40882e45d3f9 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:49.287813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1172/2326] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:36:49.292736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd6ea6-5e18-42fa-ace4-da56d8f6babb admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:49.304656 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd6ea6-5e18-42fa-ace4-da56d8f6babb admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:49.309480 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f43e9-5303-4f8e-bee5-e4fcbcd42031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:49.310370 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f43e9-5303-4f8e-bee5-e4fcbcd42031 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:49.310891 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f43e9-5303-4f8e-bee5-e4fcbcd42031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:49.311054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f43e9-5303-4f8e-bee5-e4fcbcd42031 admin admin] PATH_INFO: `/v3/projects/f96406c2d26c49bb961add3365fe1b64/users/2755db5c691f433bb04cbd78fff4f5e2/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:49.332966 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f43e9-5303-4f8e-bee5-e4fcbcd42031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96406c2d26c49bb961add3365fe1b64, user_id=2755db5c691f433bb04cbd78fff4f5e2, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:49.336504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f43e9-5303-4f8e-bee5-e4fcbcd42031 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:49.352255 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1155/2327] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:49 2022] PUT /identity/v3/projects/f96406c2d26c49bb961add3365fe1b64/users/2755db5c691f433bb04cbd78fff4f5e2/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:49.365218 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084847a-3b7a-45c4-8f01-4e642e04add4 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:49.365515 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084847a-3b7a-45c4-8f01-4e642e04add4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:49.365803 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084847a-3b7a-45c4-8f01-4e642e04add4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:49.398180 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-2084847a-3b7a-45c4-8f01-4e642e04add4 None None] MFA Rules not processed for user `2755db5c691f433bb04cbd78fff4f5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:49.399695 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-2084847a-3b7a-45c4-8f01-4e642e04add4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:49.432405 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:49.490607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1173/2328] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:49.495141 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f43e9-5303-4f8e-bee5-e4fcbcd42031 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:49.588884 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f43e9-5303-4f8e-bee5-e4fcbcd42031 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:49.595309 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27a192-e70b-498e-b268-f8aa284da900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:49.596233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27a192-e70b-498e-b268-f8aa284da900 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:49.596711 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27a192-e70b-498e-b268-f8aa284da900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:49.596992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27a192-e70b-498e-b268-f8aa284da900 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:49.597706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27a192-e70b-498e-b268-f8aa284da900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:49.599665 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27a192-e70b-498e-b268-f8aa284da900 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:49.642913 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1156/2329] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:49 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:49.648803 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084847a-3b7a-45c4-8f01-4e642e04add4 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:49.667789 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084847a-3b7a-45c4-8f01-4e642e04add4 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:49.766210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4bc80-dcc3-46be-8cb6-4c59ff289377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:49.767516 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4bc80-dcc3-46be-8cb6-4c59ff289377 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:49.767775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4bc80-dcc3-46be-8cb6-4c59ff289377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:49.768352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4bc80-dcc3-46be-8cb6-4c59ff289377 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:49.768961 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4bc80-dcc3-46be-8cb6-4c59ff289377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:49.773238 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4bc80-dcc3-46be-8cb6-4c59ff289377 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:49.798587 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1174/2330] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:49 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:36:49.803222 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27a192-e70b-498e-b268-f8aa284da900 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:49.815432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27a192-e70b-498e-b268-f8aa284da900 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:49.820475 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3e1a5-3024-4375-a5b2-1ddcf8251db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:49.821219 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3e1a5-3024-4375-a5b2-1ddcf8251db1 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:49.821574 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3e1a5-3024-4375-a5b2-1ddcf8251db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:49.822039 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3e1a5-3024-4375-a5b2-1ddcf8251db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:49.822872 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3e1a5-3024-4375-a5b2-1ddcf8251db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:49.824893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3e1a5-3024-4375-a5b2-1ddcf8251db1 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:49.846897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1157/2331] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:49.851070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4bc80-dcc3-46be-8cb6-4c59ff289377 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:49.864200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4bc80-dcc3-46be-8cb6-4c59ff289377 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:49.870357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c8858-3556-48c9-96b7-7ce853f25ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:49.871384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c8858-3556-48c9-96b7-7ce853f25ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:49.872098 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c8858-3556-48c9-96b7-7ce853f25ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:49.872706 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c8858-3556-48c9-96b7-7ce853f25ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:49.873635 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c8858-3556-48c9-96b7-7ce853f25ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:49.876203 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c8858-3556-48c9-96b7-7ce853f25ee8 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:49.895033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1175/2332] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:49.897939 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3e1a5-3024-4375-a5b2-1ddcf8251db1 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:49.923492 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3e1a5-3024-4375-a5b2-1ddcf8251db1 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:49.929573 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82df1a-dd35-4395-9430-9bc7cb27d476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:49.930771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82df1a-dd35-4395-9430-9bc7cb27d476 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:49.931183 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82df1a-dd35-4395-9430-9bc7cb27d476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:49.931569 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82df1a-dd35-4395-9430-9bc7cb27d476 admin admin] PATH_INFO: `/v3/projects/3c71a93b590541c5868a2bd93b056b84/users/73262bcc27314625abee11e37b74b280/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:49.958137 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82df1a-dd35-4395-9430-9bc7cb27d476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c71a93b590541c5868a2bd93b056b84, user_id=73262bcc27314625abee11e37b74b280, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:49.961184 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82df1a-dd35-4395-9430-9bc7cb27d476 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:49.979453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1158/2333] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:49 2022] PUT /identity/v3/projects/3c71a93b590541c5868a2bd93b056b84/users/73262bcc27314625abee11e37b74b280/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:49.989921 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f0c20-e287-4e86-8967-b5d73f3c3ef2 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:49.990359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f0c20-e287-4e86-8967-b5d73f3c3ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:49.990708 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f0c20-e287-4e86-8967-b5d73f3c3ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:50.041150 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-692f0c20-e287-4e86-8967-b5d73f3c3ef2 None None] MFA Rules not processed for user `73262bcc27314625abee11e37b74b280`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:50.041150 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-692f0c20-e287-4e86-8967-b5d73f3c3ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:50.169069 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1176/2334] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:50.174381 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82df1a-dd35-4395-9430-9bc7cb27d476 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:50.295761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:50.363297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82df1a-dd35-4395-9430-9bc7cb27d476 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:50.377442 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5384d38a-7069-462d-ab3a-789dd925036d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:50.378197 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5384d38a-7069-462d-ab3a-789dd925036d admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:50.378489 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5384d38a-7069-462d-ab3a-789dd925036d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:50.378761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5384d38a-7069-462d-ab3a-789dd925036d admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:50.379594 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5384d38a-7069-462d-ab3a-789dd925036d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:50.384713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5384d38a-7069-462d-ab3a-789dd925036d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:50.444167 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1159/2335] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:36:50 2022] POST /identity/v3/projects => generated 387 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:50.448008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f0c20-e287-4e86-8967-b5d73f3c3ef2 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:50.468937 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f0c20-e287-4e86-8967-b5d73f3c3ef2 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:50.533384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:50.568044 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece2ef8-8c80-4cad-89d0-07ca037a0b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:50.569623 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece2ef8-8c80-4cad-89d0-07ca037a0b46 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:50.570145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece2ef8-8c80-4cad-89d0-07ca037a0b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:50.570497 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece2ef8-8c80-4cad-89d0-07ca037a0b46 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:50.572523 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece2ef8-8c80-4cad-89d0-07ca037a0b46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:50.576198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece2ef8-8c80-4cad-89d0-07ca037a0b46 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:50.612742 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1177/2336] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:36:50 2022] POST /identity/v3/users => generated 300 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:36:50.616521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5384d38a-7069-462d-ab3a-789dd925036d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:50.629036 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5384d38a-7069-462d-ab3a-789dd925036d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:50.635737 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73d1ff-b9f7-4969-bb3c-92a61d059a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:50.636576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73d1ff-b9f7-4969-bb3c-92a61d059a21 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:50.636928 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73d1ff-b9f7-4969-bb3c-92a61d059a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:50.637305 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73d1ff-b9f7-4969-bb3c-92a61d059a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:50.638160 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73d1ff-b9f7-4969-bb3c-92a61d059a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:50.639931 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73d1ff-b9f7-4969-bb3c-92a61d059a21 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:50.653997 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1160/2337] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:50.658775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece2ef8-8c80-4cad-89d0-07ca037a0b46 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:50.672925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece2ef8-8c80-4cad-89d0-07ca037a0b46 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:50.705547 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2c2dd-5d2b-4cf9-9be6-d0be1bc49b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:50.706651 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2c2dd-5d2b-4cf9-9be6-d0be1bc49b40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:50.708234 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2c2dd-5d2b-4cf9-9be6-d0be1bc49b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:50.708661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2c2dd-5d2b-4cf9-9be6-d0be1bc49b40 admin admin] PATH_INFO: `/v3/projects/a202d88f3dae4f7995d63bfcc416f54c/users/c2a02e322c8a48b0a08784fc0e9fab50/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:50.732440 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2c2dd-5d2b-4cf9-9be6-d0be1bc49b40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a202d88f3dae4f7995d63bfcc416f54c, user_id=c2a02e322c8a48b0a08784fc0e9fab50, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:50.735613 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2c2dd-5d2b-4cf9-9be6-d0be1bc49b40 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:50.763733 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1178/2338] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:50 2022] PUT /identity/v3/projects/a202d88f3dae4f7995d63bfcc416f54c/users/c2a02e322c8a48b0a08784fc0e9fab50/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:50.768556 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73d1ff-b9f7-4969-bb3c-92a61d059a21 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:50.781748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73d1ff-b9f7-4969-bb3c-92a61d059a21 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:50.868373 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81657242-b9fb-490c-98b2-6cb00ec1a550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:50.869343 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81657242-b9fb-490c-98b2-6cb00ec1a550 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:50.869819 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81657242-b9fb-490c-98b2-6cb00ec1a550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:50.870549 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81657242-b9fb-490c-98b2-6cb00ec1a550 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:50.871458 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81657242-b9fb-490c-98b2-6cb00ec1a550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:50.873954 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81657242-b9fb-490c-98b2-6cb00ec1a550 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:50.899869 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1161/2339] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:36:50.904077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2c2dd-5d2b-4cf9-9be6-d0be1bc49b40 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:50.918968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2c2dd-5d2b-4cf9-9be6-d0be1bc49b40 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:50.935683 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbb5dc-cc67-47a2-b5d3-106cbb93af87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:50.937141 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbb5dc-cc67-47a2-b5d3-106cbb93af87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:50.937576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbb5dc-cc67-47a2-b5d3-106cbb93af87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:50.938010 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbb5dc-cc67-47a2-b5d3-106cbb93af87 admin admin] PATH_INFO: `/v3/projects/a202d88f3dae4f7995d63bfcc416f54c/users/c2a02e322c8a48b0a08784fc0e9fab50/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:50.944873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbb5dc-cc67-47a2-b5d3-106cbb93af87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a202d88f3dae4f7995d63bfcc416f54c, user_id=c2a02e322c8a48b0a08784fc0e9fab50, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:50.947575 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbb5dc-cc67-47a2-b5d3-106cbb93af87 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:50.986987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1179/2340] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:36:50 2022] PUT /identity/v3/projects/a202d88f3dae4f7995d63bfcc416f54c/users/c2a02e322c8a48b0a08784fc0e9fab50/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:50.997668 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a450e-e804-486d-9986-ff3c7f774a61 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:50.997968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a450e-e804-486d-9986-ff3c7f774a61 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:50.998239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a450e-e804-486d-9986-ff3c7f774a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:51.031323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f53a450e-e804-486d-9986-ff3c7f774a61 None None] MFA Rules not processed for user `c2a02e322c8a48b0a08784fc0e9fab50`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:51.033415 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f53a450e-e804-486d-9986-ff3c7f774a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:51.165058 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1162/2341] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:51.230304 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbb5dc-cc67-47a2-b5d3-106cbb93af87 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:51.236973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-dabbb5dc-cc67-47a2-b5d3-106cbb93af87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:51.329604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbb5dc-cc67-47a2-b5d3-106cbb93af87 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:51.337106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75023083-2597-4efa-af24-b3fd7f3bb853 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:51.337934 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75023083-2597-4efa-af24-b3fd7f3bb853 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:51.338316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75023083-2597-4efa-af24-b3fd7f3bb853 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:51.338718 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75023083-2597-4efa-af24-b3fd7f3bb853 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:51.352891 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75023083-2597-4efa-af24-b3fd7f3bb853 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:51.355075 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75023083-2597-4efa-af24-b3fd7f3bb853 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:51.373896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1180/2342] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:52.025597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a450e-e804-486d-9986-ff3c7f774a61 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:52.027127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f53a450e-e804-486d-9986-ff3c7f774a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:52.039439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a450e-e804-486d-9986-ff3c7f774a61 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:52.043087 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c4312-e764-47f1-8e38-2149fbbe15d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:52.043829 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c4312-e764-47f1-8e38-2149fbbe15d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:52.044173 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c4312-e764-47f1-8e38-2149fbbe15d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:52.044509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c4312-e764-47f1-8e38-2149fbbe15d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:52.090015 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:52.119003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c4312-e764-47f1-8e38-2149fbbe15d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:52.120468 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c4312-e764-47f1-8e38-2149fbbe15d2 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:52.130400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1163/2343] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:55.896805 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96773c2c-7866-44c9-8d38-cd6c54e9cb5d None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:55.898746 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96773c2c-7866-44c9-8d38-cd6c54e9cb5d None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:55.899626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96773c2c-7866-44c9-8d38-cd6c54e9cb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:55.935961 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-96773c2c-7866-44c9-8d38-cd6c54e9cb5d None None] MFA Rules not processed for user `4f75b8b484384554ad59aa20535875e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:55.938116 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-96773c2c-7866-44c9-8d38-cd6c54e9cb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:56.015335 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1181/2344] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 14 20:36:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:56.073978 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c4312-e764-47f1-8e38-2149fbbe15d2 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:56.076344 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-193c4312-e764-47f1-8e38-2149fbbe15d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:56.160163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c4312-e764-47f1-8e38-2149fbbe15d2 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:56.166186 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90169654-2605-4bfd-bd6b-b895b111384e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc23233929144a5b7b3a1733da19828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:56.166966 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90169654-2605-4bfd-bd6b-b895b111384e service swift] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:56.167292 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90169654-2605-4bfd-bd6b-b895b111384e service swift] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:56.167523 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90169654-2605-4bfd-bd6b-b895b111384e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:56.187004 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90169654-2605-4bfd-bd6b-b895b111384e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:56.189263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90169654-2605-4bfd-bd6b-b895b111384e service swift] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:56.199132 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1164/2345] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 14 20:36:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 20:36:56.767704 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5bbcc-88f9-466a-94fb-1a589c4083e9 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:56.768025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5bbcc-88f9-466a-94fb-1a589c4083e9 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:56.768303 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5bbcc-88f9-466a-94fb-1a589c4083e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:56.789761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-5cb5bbcc-88f9-466a-94fb-1a589c4083e9 None None] MFA Rules not processed for user `4f75b8b484384554ad59aa20535875e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:56.791390 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-5cb5bbcc-88f9-466a-94fb-1a589c4083e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:56.800524 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1182/2346] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jul 14 20:36:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:56.813493 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90169654-2605-4bfd-bd6b-b895b111384e service swift] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:56.824338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90169654-2605-4bfd-bd6b-b895b111384e service swift] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:56.829514 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3460771-2617-40e2-9221-ca2ed85a3d55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc23233929144a5b7b3a1733da19828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:56.830156 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3460771-2617-40e2-9221-ca2ed85a3d55 service swift] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:56.830445 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3460771-2617-40e2-9221-ca2ed85a3d55 service swift] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:56.830723 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3460771-2617-40e2-9221-ca2ed85a3d55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:56.838950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3460771-2617-40e2-9221-ca2ed85a3d55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:56.840557 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3460771-2617-40e2-9221-ca2ed85a3d55 service swift] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:56.849285 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1165/2347] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 14 20:36:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 20:36:57.111638 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8efee1-830b-4435-8bd0-f25e5b301a9c None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:57.112166 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8efee1-830b-4435-8bd0-f25e5b301a9c None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:57.112260 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8efee1-830b-4435-8bd0-f25e5b301a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:57.131709 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-1f8efee1-830b-4435-8bd0-f25e5b301a9c None None] MFA Rules not processed for user `4f75b8b484384554ad59aa20535875e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:57.132952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-1f8efee1-830b-4435-8bd0-f25e5b301a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:57.140518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1183/2348] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 14 20:36:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:57.153496 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3460771-2617-40e2-9221-ca2ed85a3d55 service swift] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:57.165671 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3460771-2617-40e2-9221-ca2ed85a3d55 service swift] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:57.173625 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbf72b-dc30-4aad-83f7-584fc2116ab3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc23233929144a5b7b3a1733da19828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:57.174295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbf72b-dc30-4aad-83f7-584fc2116ab3 service swift] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:57.174577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbf72b-dc30-4aad-83f7-584fc2116ab3 service swift] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:57.174853 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbf72b-dc30-4aad-83f7-584fc2116ab3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:57.184180 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bbf72b-dc30-4aad-83f7-584fc2116ab3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:57.185810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bbf72b-dc30-4aad-83f7-584fc2116ab3 service swift] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:57.196928 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1166/2349] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 14 20:36:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 20:36:57.462492 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8efee1-830b-4435-8bd0-f25e5b301a9c None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:57.473743 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8efee1-830b-4435-8bd0-f25e5b301a9c None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:57.477366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e484d0d-9727-4e05-8698-283152d9fe34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:57.478007 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e484d0d-9727-4e05-8698-283152d9fe34 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:57.478289 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e484d0d-9727-4e05-8698-283152d9fe34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:57.478565 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e484d0d-9727-4e05-8698-283152d9fe34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:57.500172 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:57.547915 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e484d0d-9727-4e05-8698-283152d9fe34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:57.549405 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e484d0d-9727-4e05-8698-283152d9fe34 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:57.553378 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67ae0f-b381-46bb-b342-a6a7216214c0 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:57.554049 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67ae0f-b381-46bb-b342-a6a7216214c0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:57.555280 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67ae0f-b381-46bb-b342-a6a7216214c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:57.560360 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1184/2350] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:36:57.596711 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8b67ae0f-b381-46bb-b342-a6a7216214c0 None None] MFA Rules not processed for user `4f75b8b484384554ad59aa20535875e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:57.597947 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8b67ae0f-b381-46bb-b342-a6a7216214c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:57.620417 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1167/2351] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jul 14 20:36:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:36:57.621463 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e484d0d-9727-4e05-8698-283152d9fe34 service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:57.654539 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e484d0d-9727-4e05-8698-283152d9fe34 service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:57.667211 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165276f2-e9e7-4798-a04d-92dad92e3da3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc23233929144a5b7b3a1733da19828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:57.668356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165276f2-e9e7-4798-a04d-92dad92e3da3 service swift] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:57.668880 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165276f2-e9e7-4798-a04d-92dad92e3da3 service swift] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:57.669370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165276f2-e9e7-4798-a04d-92dad92e3da3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:57.680795 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165276f2-e9e7-4798-a04d-92dad92e3da3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:57.683075 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165276f2-e9e7-4798-a04d-92dad92e3da3 service swift] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:57.704620 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1185/2352] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 14 20:36:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 20:36:59.339774 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67ae0f-b381-46bb-b342-a6a7216214c0 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:59.343348 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8b67ae0f-b381-46bb-b342-a6a7216214c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:59.415758 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67ae0f-b381-46bb-b342-a6a7216214c0 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:59.420102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e573c-a859-4a2e-928c-c857dae07073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:59.420720 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e573c-a859-4a2e-928c-c857dae07073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:59.420996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e573c-a859-4a2e-928c-c857dae07073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:59.421271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e573c-a859-4a2e-928c-c857dae07073 admin admin] PATH_INFO: `/v3/users/3a740dea2e494a4d80d1b428a373e930` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:59.422559 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e573c-a859-4a2e-928c-c857dae07073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a740dea2e494a4d80d1b428a373e930)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:59.424843 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e573c-a859-4a2e-928c-c857dae07073 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:59.465868 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:36:59.485329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ef6e573c-a859-4a2e-928c-c857dae07073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a740dea2e494a4d80d1b428a373e930'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:59.495424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ef6e573c-a859-4a2e-928c-c857dae07073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a740dea2e494a4d80d1b428a373e930'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:59.496474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ef6e573c-a859-4a2e-928c-c857dae07073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a740dea2e494a4d80d1b428a373e930'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:59.497149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ef6e573c-a859-4a2e-928c-c857dae07073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a740dea2e494a4d80d1b428a373e930'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:59.510083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ef6e573c-a859-4a2e-928c-c857dae07073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a740dea2e494a4d80d1b428a373e930'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:36:59.533943 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1168/2353] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:36:59 2022] DELETE /identity/v3/users/3a740dea2e494a4d80d1b428a373e930 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:36:59.538786 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4c6a5-9b2a-440f-b995-560f716ac4b8 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:59.539267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4c6a5-9b2a-440f-b995-560f716ac4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:59.539488 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4c6a5-9b2a-440f-b995-560f716ac4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:59.560860 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-d7f4c6a5-9b2a-440f-b995-560f716ac4b8 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:36:59.562098 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d7f4c6a5-9b2a-440f-b995-560f716ac4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:59.655917 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1186/2354] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:36:59.674063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e573c-a859-4a2e-928c-c857dae07073 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:36:59.676665 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ef6e573c-a859-4a2e-928c-c857dae07073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:36:59.813040 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e573c-a859-4a2e-928c-c857dae07073 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:36:59.828660 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcc059-bb22-4521-9d3d-eebbb977ab03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:36:59.830377 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcc059-bb22-4521-9d3d-eebbb977ab03 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:36:59.831023 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcc059-bb22-4521-9d3d-eebbb977ab03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:36:59.831539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcc059-bb22-4521-9d3d-eebbb977ab03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:36:59.851956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fcc059-bb22-4521-9d3d-eebbb977ab03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:36:59.855463 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fcc059-bb22-4521-9d3d-eebbb977ab03 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:36:59.877818 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1169/2355] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:36:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:37:00.238879 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4c6a5-9b2a-440f-b995-560f716ac4b8 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:00.240871 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d7f4c6a5-9b2a-440f-b995-560f716ac4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:00.267677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4c6a5-9b2a-440f-b995-560f716ac4b8 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:00.283835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e058671-07fc-4c78-84ac-a9ee8dc86f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:00.284754 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e058671-07fc-4c78-84ac-a9ee8dc86f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:00.285043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e058671-07fc-4c78-84ac-a9ee8dc86f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:00.285973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e058671-07fc-4c78-84ac-a9ee8dc86f04 admin admin] PATH_INFO: `/v3/users/c80784a1d58a4b9186636b37d186a739` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:00.290401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e058671-07fc-4c78-84ac-a9ee8dc86f04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c80784a1d58a4b9186636b37d186a739)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:00.293626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e058671-07fc-4c78-84ac-a9ee8dc86f04 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:00.401291 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-9e058671-07fc-4c78-84ac-a9ee8dc86f04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c80784a1d58a4b9186636b37d186a739'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:00.437094 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcc059-bb22-4521-9d3d-eebbb977ab03 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:00.439624 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a2fcc059-bb22-4521-9d3d-eebbb977ab03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:00.452964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-9e058671-07fc-4c78-84ac-a9ee8dc86f04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c80784a1d58a4b9186636b37d186a739'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:00.465392 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcc059-bb22-4521-9d3d-eebbb977ab03 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:00.476251 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82709068-dafa-41c3-acdc-cc9b21dc3935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:00.477405 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82709068-dafa-41c3-acdc-cc9b21dc3935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:00.477844 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82709068-dafa-41c3-acdc-cc9b21dc3935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:00.478265 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82709068-dafa-41c3-acdc-cc9b21dc3935 admin admin] PATH_INFO: `/v3/projects/03288f0649a848fca16eda063bc9817e` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:00.479973 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82709068-dafa-41c3-acdc-cc9b21dc3935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03288f0649a848fca16eda063bc9817e)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:00.482153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82709068-dafa-41c3-acdc-cc9b21dc3935 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:00.483723 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-9e058671-07fc-4c78-84ac-a9ee8dc86f04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c80784a1d58a4b9186636b37d186a739'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:00.484869 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-9e058671-07fc-4c78-84ac-a9ee8dc86f04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c80784a1d58a4b9186636b37d186a739'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:00.485994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-9e058671-07fc-4c78-84ac-a9ee8dc86f04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c80784a1d58a4b9186636b37d186a739'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:00.522620 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1187/2356] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:37:00 2022] DELETE /identity/v3/users/c80784a1d58a4b9186636b37d186a739 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:37:00.522620 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e058671-07fc-4c78-84ac-a9ee8dc86f04 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:00.523802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9e058671-07fc-4c78-84ac-a9ee8dc86f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:00.545130 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-82709068-dafa-41c3-acdc-cc9b21dc3935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03288f0649a848fca16eda063bc9817e', 'request_id': 'req-82709068-dafa-41c3-acdc-cc9b21dc3935'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:00.547829 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:37:00.554823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-82709068-dafa-41c3-acdc-cc9b21dc3935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03288f0649a848fca16eda063bc9817e', 'request_id': 'req-82709068-dafa-41c3-acdc-cc9b21dc3935'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:00.562561 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-82709068-dafa-41c3-acdc-cc9b21dc3935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03288f0649a848fca16eda063bc9817e', 'request_id': 'req-82709068-dafa-41c3-acdc-cc9b21dc3935'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:00.579604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-82709068-dafa-41c3-acdc-cc9b21dc3935 admin admin] The token cache is being invalidate because project 03288f0649a848fca16eda063bc9817e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:37:00.579903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-82709068-dafa-41c3-acdc-cc9b21dc3935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:00.580546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-82709068-dafa-41c3-acdc-cc9b21dc3935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:00.582673 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1170/2357] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:37:00 2022] DELETE /identity/v3/projects/03288f0649a848fca16eda063bc9817e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:37:00.585210 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c04745-af63-4a65-aff8-a1b6fa298508 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:00.585594 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c04745-af63-4a65-aff8-a1b6fa298508 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:00.586213 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c04745-af63-4a65-aff8-a1b6fa298508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:00.610293 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-01c04745-af63-4a65-aff8-a1b6fa298508 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:37:00.610531 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e058671-07fc-4c78-84ac-a9ee8dc86f04 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:00.611602 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-01c04745-af63-4a65-aff8-a1b6fa298508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:00.652917 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:37:00.693377 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1171/2358] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:37:00.705633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0ff3a-4bc7-457b-909c-dbc6e215139a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:00.706376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c04745-af63-4a65-aff8-a1b6fa298508 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:00.706806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0ff3a-4bc7-457b-909c-dbc6e215139a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:00.707222 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0ff3a-4bc7-457b-909c-dbc6e215139a admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:00.707603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0ff3a-4bc7-457b-909c-dbc6e215139a admin admin] PATH_INFO: `/v3/users/2755db5c691f433bb04cbd78fff4f5e2` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:00.707959 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-01c04745-af63-4a65-aff8-a1b6fa298508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:00.709152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0ff3a-4bc7-457b-909c-dbc6e215139a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2755db5c691f433bb04cbd78fff4f5e2)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:00.713548 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0ff3a-4bc7-457b-909c-dbc6e215139a admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:00.803574 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-64a0ff3a-4bc7-457b-909c-dbc6e215139a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2755db5c691f433bb04cbd78fff4f5e2'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:00.807041 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c04745-af63-4a65-aff8-a1b6fa298508 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:00.813851 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d8e8a-b5b4-4e4e-83a0-44cbab929651 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:00.814581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d8e8a-b5b4-4e4e-83a0-44cbab929651 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:00.815067 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d8e8a-b5b4-4e4e-83a0-44cbab929651 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:00.815370 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d8e8a-b5b4-4e4e-83a0-44cbab929651 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:00.815865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-64a0ff3a-4bc7-457b-909c-dbc6e215139a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2755db5c691f433bb04cbd78fff4f5e2'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:00.823789 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-64a0ff3a-4bc7-457b-909c-dbc6e215139a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2755db5c691f433bb04cbd78fff4f5e2'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:00.824712 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-64a0ff3a-4bc7-457b-909c-dbc6e215139a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2755db5c691f433bb04cbd78fff4f5e2'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:00.825417 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-64a0ff3a-4bc7-457b-909c-dbc6e215139a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2755db5c691f433bb04cbd78fff4f5e2'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:00.830445 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d8e8a-b5b4-4e4e-83a0-44cbab929651 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:00.833350 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d8e8a-b5b4-4e4e-83a0-44cbab929651 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:00.845282 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1172/2359] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:37:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:37:00.848584 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1188/2360] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:37:00 2022] DELETE /identity/v3/users/2755db5c691f433bb04cbd78fff4f5e2 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:37:00.852963 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68961cc2-807a-4cca-b128-a7e1a6426228 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:00.853348 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68961cc2-807a-4cca-b128-a7e1a6426228 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:00.853707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68961cc2-807a-4cca-b128-a7e1a6426228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:00.874345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-68961cc2-807a-4cca-b128-a7e1a6426228 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:37:00.876418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-68961cc2-807a-4cca-b128-a7e1a6426228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:00.972746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1173/2361] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:37:00.983407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0ff3a-4bc7-457b-909c-dbc6e215139a admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:00.985797 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-64a0ff3a-4bc7-457b-909c-dbc6e215139a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:01.065955 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:37:01.078360 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0ff3a-4bc7-457b-909c-dbc6e215139a admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:01.082703 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8a0b7-298c-4854-a949-e2d2acab8603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:01.086476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8a0b7-298c-4854-a949-e2d2acab8603 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:01.086898 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8a0b7-298c-4854-a949-e2d2acab8603 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:01.087319 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8a0b7-298c-4854-a949-e2d2acab8603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:01.097031 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68961cc2-807a-4cca-b128-a7e1a6426228 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:01.098615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-68961cc2-807a-4cca-b128-a7e1a6426228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:01.102614 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b8a0b7-298c-4854-a949-e2d2acab8603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:01.104547 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b8a0b7-298c-4854-a949-e2d2acab8603 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:01.113477 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68961cc2-807a-4cca-b128-a7e1a6426228 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:01.119296 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57540c83-9588-4608-ad6f-43d666c49411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:01.119843 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1189/2362] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:37:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:37:01.131320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57540c83-9588-4608-ad6f-43d666c49411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:01.131717 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57540c83-9588-4608-ad6f-43d666c49411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:01.132017 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57540c83-9588-4608-ad6f-43d666c49411 admin admin] PATH_INFO: `/v3/projects/3c1e417367fe41c9943a8b2297742120` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:01.133470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57540c83-9588-4608-ad6f-43d666c49411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c1e417367fe41c9943a8b2297742120)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:01.135427 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57540c83-9588-4608-ad6f-43d666c49411 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:01.199195 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-57540c83-9588-4608-ad6f-43d666c49411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c1e417367fe41c9943a8b2297742120', 'request_id': 'req-57540c83-9588-4608-ad6f-43d666c49411'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:01.206384 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-57540c83-9588-4608-ad6f-43d666c49411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c1e417367fe41c9943a8b2297742120', 'request_id': 'req-57540c83-9588-4608-ad6f-43d666c49411'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:01.217453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-57540c83-9588-4608-ad6f-43d666c49411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c1e417367fe41c9943a8b2297742120', 'request_id': 'req-57540c83-9588-4608-ad6f-43d666c49411'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:01.229330 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:37:01.233694 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-57540c83-9588-4608-ad6f-43d666c49411 admin admin] The token cache is being invalidate because project 3c1e417367fe41c9943a8b2297742120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:37:01.234394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-57540c83-9588-4608-ad6f-43d666c49411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:01.236898 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-57540c83-9588-4608-ad6f-43d666c49411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:01.251374 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1174/2363] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:37:01 2022] DELETE /identity/v3/projects/3c1e417367fe41c9943a8b2297742120 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:37:01.556220 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8a0b7-298c-4854-a949-e2d2acab8603 service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:01.558546 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-a4b8a0b7-298c-4854-a949-e2d2acab8603 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:01.573528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8a0b7-298c-4854-a949-e2d2acab8603 service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:01.654685 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c7854-c7fe-411b-9194-e29efda80cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:01.655681 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c7854-c7fe-411b-9194-e29efda80cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:01.655959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c7854-c7fe-411b-9194-e29efda80cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:01.656234 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c7854-c7fe-411b-9194-e29efda80cae admin admin] PATH_INFO: `/v3/projects/f96406c2d26c49bb961add3365fe1b64` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:01.658005 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c7854-c7fe-411b-9194-e29efda80cae admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96406c2d26c49bb961add3365fe1b64)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:01.661064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c7854-c7fe-411b-9194-e29efda80cae admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:01.710661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7a4c7854-c7fe-411b-9194-e29efda80cae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96406c2d26c49bb961add3365fe1b64', 'request_id': 'req-7a4c7854-c7fe-411b-9194-e29efda80cae'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:01.719381 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7a4c7854-c7fe-411b-9194-e29efda80cae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96406c2d26c49bb961add3365fe1b64', 'request_id': 'req-7a4c7854-c7fe-411b-9194-e29efda80cae'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:01.726961 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7a4c7854-c7fe-411b-9194-e29efda80cae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96406c2d26c49bb961add3365fe1b64', 'request_id': 'req-7a4c7854-c7fe-411b-9194-e29efda80cae'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:01.737673 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7a4c7854-c7fe-411b-9194-e29efda80cae admin admin] The token cache is being invalidate because project f96406c2d26c49bb961add3365fe1b64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:37:01.737963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7a4c7854-c7fe-411b-9194-e29efda80cae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:01.738628 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-7a4c7854-c7fe-411b-9194-e29efda80cae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:01.740189 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1190/2364] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:37:01 2022] DELETE /identity/v3/projects/f96406c2d26c49bb961add3365fe1b64 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:37:01.744081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57540c83-9588-4608-ad6f-43d666c49411 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:01.745509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-57540c83-9588-4608-ad6f-43d666c49411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:01.759787 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57540c83-9588-4608-ad6f-43d666c49411 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:01.817426 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650c0c3-6774-4390-b304-6351531ae5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:01.818223 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650c0c3-6774-4390-b304-6351531ae5c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:01.818505 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650c0c3-6774-4390-b304-6351531ae5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:01.818777 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650c0c3-6774-4390-b304-6351531ae5c3 admin admin] PATH_INFO: `/v3/users/73262bcc27314625abee11e37b74b280` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:01.819998 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650c0c3-6774-4390-b304-6351531ae5c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73262bcc27314625abee11e37b74b280)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:01.821924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650c0c3-6774-4390-b304-6351531ae5c3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:01.868163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-6650c0c3-6774-4390-b304-6351531ae5c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73262bcc27314625abee11e37b74b280'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:01.876543 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-6650c0c3-6774-4390-b304-6351531ae5c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73262bcc27314625abee11e37b74b280'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:01.877128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-6650c0c3-6774-4390-b304-6351531ae5c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73262bcc27314625abee11e37b74b280'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:01.877733 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-6650c0c3-6774-4390-b304-6351531ae5c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73262bcc27314625abee11e37b74b280'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:01.890752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-6650c0c3-6774-4390-b304-6351531ae5c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73262bcc27314625abee11e37b74b280'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:01.904039 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1175/2365] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:37:01 2022] DELETE /identity/v3/users/73262bcc27314625abee11e37b74b280 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:37:02.206172 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c7854-c7fe-411b-9194-e29efda80cae admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:02.207616 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-7a4c7854-c7fe-411b-9194-e29efda80cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:02.220384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c7854-c7fe-411b-9194-e29efda80cae admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:02.284485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7602e-35d8-4bfa-8cd9-a8ebb3856a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:02.285363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7602e-35d8-4bfa-8cd9-a8ebb3856a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:02.285648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7602e-35d8-4bfa-8cd9-a8ebb3856a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:02.285926 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7602e-35d8-4bfa-8cd9-a8ebb3856a3b admin admin] PATH_INFO: `/v3/projects/3c71a93b590541c5868a2bd93b056b84` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:02.287158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7602e-35d8-4bfa-8cd9-a8ebb3856a3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c71a93b590541c5868a2bd93b056b84)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:02.289065 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7602e-35d8-4bfa-8cd9-a8ebb3856a3b admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:02.311746 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:37:02.337588 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-76f7602e-35d8-4bfa-8cd9-a8ebb3856a3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c71a93b590541c5868a2bd93b056b84', 'request_id': 'req-76f7602e-35d8-4bfa-8cd9-a8ebb3856a3b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:02.344878 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-76f7602e-35d8-4bfa-8cd9-a8ebb3856a3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c71a93b590541c5868a2bd93b056b84', 'request_id': 'req-76f7602e-35d8-4bfa-8cd9-a8ebb3856a3b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:02.352059 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-76f7602e-35d8-4bfa-8cd9-a8ebb3856a3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c71a93b590541c5868a2bd93b056b84', 'request_id': 'req-76f7602e-35d8-4bfa-8cd9-a8ebb3856a3b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:02.364608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-76f7602e-35d8-4bfa-8cd9-a8ebb3856a3b admin admin] The token cache is being invalidate because project 3c71a93b590541c5868a2bd93b056b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:37:02.364996 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-76f7602e-35d8-4bfa-8cd9-a8ebb3856a3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:02.365692 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-76f7602e-35d8-4bfa-8cd9-a8ebb3856a3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:02.367806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1191/2366] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:37:02 2022] DELETE /identity/v3/projects/3c71a93b590541c5868a2bd93b056b84 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:37:02.371982 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650c0c3-6774-4390-b304-6351531ae5c3 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:02.373466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6650c0c3-6774-4390-b304-6351531ae5c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:02.387826 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650c0c3-6774-4390-b304-6351531ae5c3 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:02.456169 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8376d-963c-4041-93ca-fd024e392ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:02.457047 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8376d-963c-4041-93ca-fd024e392ab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:02.457329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8376d-963c-4041-93ca-fd024e392ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:02.457605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8376d-963c-4041-93ca-fd024e392ab3 admin admin] PATH_INFO: `/v3/users/c2a02e322c8a48b0a08784fc0e9fab50` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:02.458793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf8376d-963c-4041-93ca-fd024e392ab3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2a02e322c8a48b0a08784fc0e9fab50)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:02.460750 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf8376d-963c-4041-93ca-fd024e392ab3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:02.481768 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:37:02.507217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ccf8376d-963c-4041-93ca-fd024e392ab3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2a02e322c8a48b0a08784fc0e9fab50'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:02.514626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ccf8376d-963c-4041-93ca-fd024e392ab3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2a02e322c8a48b0a08784fc0e9fab50'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:02.515434 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ccf8376d-963c-4041-93ca-fd024e392ab3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2a02e322c8a48b0a08784fc0e9fab50'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:02.516075 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ccf8376d-963c-4041-93ca-fd024e392ab3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2a02e322c8a48b0a08784fc0e9fab50'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:02.528092 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ccf8376d-963c-4041-93ca-fd024e392ab3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2a02e322c8a48b0a08784fc0e9fab50'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:02.540022 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1176/2367] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:37:02 2022] DELETE /identity/v3/users/c2a02e322c8a48b0a08784fc0e9fab50 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:37:02.979679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7602e-35d8-4bfa-8cd9-a8ebb3856a3b admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:02.991969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-76f7602e-35d8-4bfa-8cd9-a8ebb3856a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:03.025625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7602e-35d8-4bfa-8cd9-a8ebb3856a3b admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:03.128993 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeeee3b-35fc-41ed-bbca-2a186cd98d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:03.130236 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeeee3b-35fc-41ed-bbca-2a186cd98d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:03.130236 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeeee3b-35fc-41ed-bbca-2a186cd98d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:03.130532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeeee3b-35fc-41ed-bbca-2a186cd98d69 admin admin] PATH_INFO: `/v3/projects/a202d88f3dae4f7995d63bfcc416f54c` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:03.131840 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeeee3b-35fc-41ed-bbca-2a186cd98d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a202d88f3dae4f7995d63bfcc416f54c)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:03.133891 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeeee3b-35fc-41ed-bbca-2a186cd98d69 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:03.206578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cbeeee3b-35fc-41ed-bbca-2a186cd98d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a202d88f3dae4f7995d63bfcc416f54c', 'request_id': 'req-cbeeee3b-35fc-41ed-bbca-2a186cd98d69'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:03.217370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cbeeee3b-35fc-41ed-bbca-2a186cd98d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a202d88f3dae4f7995d63bfcc416f54c', 'request_id': 'req-cbeeee3b-35fc-41ed-bbca-2a186cd98d69'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:03.225627 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cbeeee3b-35fc-41ed-bbca-2a186cd98d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a202d88f3dae4f7995d63bfcc416f54c', 'request_id': 'req-cbeeee3b-35fc-41ed-bbca-2a186cd98d69'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:03.239697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cbeeee3b-35fc-41ed-bbca-2a186cd98d69 admin admin] The token cache is being invalidate because project a202d88f3dae4f7995d63bfcc416f54c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:37:03.239992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cbeeee3b-35fc-41ed-bbca-2a186cd98d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:03.240712 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cbeeee3b-35fc-41ed-bbca-2a186cd98d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:37:03.242992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1192/2368] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:37:02 2022] DELETE /identity/v3/projects/a202d88f3dae4f7995d63bfcc416f54c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:37:03.249499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382fc06-b36a-41b2-8cae-36a81fb80944 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:03.250199 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382fc06-b36a-41b2-8cae-36a81fb80944 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:03.250199 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382fc06-b36a-41b2-8cae-36a81fb80944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:03.270744 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-7382fc06-b36a-41b2-8cae-36a81fb80944 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:37:03.272561 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7382fc06-b36a-41b2-8cae-36a81fb80944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:03.351231 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1177/2369] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:37:03.357081 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112b5df-6cef-4b38-91bb-2bc9a2b6151d None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:03.357363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112b5df-6cef-4b38-91bb-2bc9a2b6151d None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:03.357637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112b5df-6cef-4b38-91bb-2bc9a2b6151d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:03.377495 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-3112b5df-6cef-4b38-91bb-2bc9a2b6151d None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:37:03.378623 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3112b5df-6cef-4b38-91bb-2bc9a2b6151d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:03.386607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1193/2370] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:37:03.391017 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382fc06-b36a-41b2-8cae-36a81fb80944 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:03.401129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382fc06-b36a-41b2-8cae-36a81fb80944 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:03.405981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572f0cd-3f67-45b4-b5e9-7fbbc66a991f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:03.406645 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572f0cd-3f67-45b4-b5e9-7fbbc66a991f admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:03.406980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572f0cd-3f67-45b4-b5e9-7fbbc66a991f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:03.407259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572f0cd-3f67-45b4-b5e9-7fbbc66a991f admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:03.407992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572f0cd-3f67-45b4-b5e9-7fbbc66a991f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:37:03.408300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572f0cd-3f67-45b4-b5e9-7fbbc66a991f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:03.409908 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572f0cd-3f67-45b4-b5e9-7fbbc66a991f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:03.423164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1178/2371] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:37:03.428229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112b5df-6cef-4b38-91bb-2bc9a2b6151d None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:03.443866 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112b5df-6cef-4b38-91bb-2bc9a2b6151d None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:03.449571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28412bf8-158c-4b5a-9074-55d8fea23d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:03.450235 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28412bf8-158c-4b5a-9074-55d8fea23d19 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:03.450519 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28412bf8-158c-4b5a-9074-55d8fea23d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:03.450796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28412bf8-158c-4b5a-9074-55d8fea23d19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:03.451548 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28412bf8-158c-4b5a-9074-55d8fea23d19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:03.453461 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28412bf8-158c-4b5a-9074-55d8fea23d19 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:03.482057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1194/2372] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:37:03 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:37:03.486845 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572f0cd-3f67-45b4-b5e9-7fbbc66a991f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:03.497054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572f0cd-3f67-45b4-b5e9-7fbbc66a991f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:03.559582 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900685d-3da1-493c-b01d-88abac655ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:03.560262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900685d-3da1-493c-b01d-88abac655ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:03.560551 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900685d-3da1-493c-b01d-88abac655ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:03.560837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900685d-3da1-493c-b01d-88abac655ab7 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:03.561532 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900685d-3da1-493c-b01d-88abac655ab7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:03.563547 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900685d-3da1-493c-b01d-88abac655ab7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:03.585533 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1179/2373] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:37:03 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:37:03.589919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28412bf8-158c-4b5a-9074-55d8fea23d19 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:03.601486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28412bf8-158c-4b5a-9074-55d8fea23d19 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:03.606747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbabd84-27b1-426e-a36c-4ecda788273d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:03.607536 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbabd84-27b1-426e-a36c-4ecda788273d admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:03.607824 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbabd84-27b1-426e-a36c-4ecda788273d admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:03.608107 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbabd84-27b1-426e-a36c-4ecda788273d admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:03.608853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbabd84-27b1-426e-a36c-4ecda788273d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:03.611300 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbabd84-27b1-426e-a36c-4ecda788273d admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:03.618081 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:37:03.625209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1195/2374] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:37:03.629982 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900685d-3da1-493c-b01d-88abac655ab7 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:03.640388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900685d-3da1-493c-b01d-88abac655ab7 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:03.645571 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed2319-4a84-4311-b2c5-fb6bd3dd8bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:03.646527 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed2319-4a84-4311-b2c5-fb6bd3dd8bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:03.646805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed2319-4a84-4311-b2c5-fb6bd3dd8bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:03.647137 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed2319-4a84-4311-b2c5-fb6bd3dd8bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:03.647885 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ed2319-4a84-4311-b2c5-fb6bd3dd8bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:03.649507 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ed2319-4a84-4311-b2c5-fb6bd3dd8bd7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:03.661316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1180/2375] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:37:03.665618 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbabd84-27b1-426e-a36c-4ecda788273d admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:03.680766 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbabd84-27b1-426e-a36c-4ecda788273d admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:03.687342 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5d845-d5fa-49e2-b398-a9fb4697053a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:03.688267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5d845-d5fa-49e2-b398-a9fb4697053a admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:03.688552 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5d845-d5fa-49e2-b398-a9fb4697053a admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:03.688835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5d845-d5fa-49e2-b398-a9fb4697053a admin admin] PATH_INFO: `/v3/projects/806c059a19044c6ebcd8f2cc50d9591d/users/1c8fa181da604f8cb41837dd570f9510/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:03.704686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5d845-d5fa-49e2-b398-a9fb4697053a admin admin] RBAC: Authorizing `identity:create_grant(project_id=806c059a19044c6ebcd8f2cc50d9591d, user_id=1c8fa181da604f8cb41837dd570f9510, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:03.707424 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5d845-d5fa-49e2-b398-a9fb4697053a admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:03.722047 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1196/2376] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:37:03 2022] PUT /identity/v3/projects/806c059a19044c6ebcd8f2cc50d9591d/users/1c8fa181da604f8cb41837dd570f9510/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:37:03.727867 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9b209-89bd-42f7-ba24-5111e586d657 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:03.728155 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9b209-89bd-42f7-ba24-5111e586d657 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:03.728429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9b209-89bd-42f7-ba24-5111e586d657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:03.736001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:37:03.748035 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-6ca9b209-89bd-42f7-ba24-5111e586d657 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:37:03.749628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6ca9b209-89bd-42f7-ba24-5111e586d657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:03.818524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1181/2377] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:37:03.828391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5d845-d5fa-49e2-b398-a9fb4697053a admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:03.830153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-dfd5d845-d5fa-49e2-b398-a9fb4697053a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:03.910673 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5d845-d5fa-49e2-b398-a9fb4697053a admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:03.915253 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd1406-fb41-4df2-bb70-6f01bb657806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:03.916058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd1406-fb41-4df2-bb70-6f01bb657806 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:03.916452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd1406-fb41-4df2-bb70-6f01bb657806 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:03.916836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd1406-fb41-4df2-bb70-6f01bb657806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:03.929173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd1406-fb41-4df2-bb70-6f01bb657806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:03.930674 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd1406-fb41-4df2-bb70-6f01bb657806 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:03.941933 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1197/2378] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:37:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:37:04.758293 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591128d-54fd-4be2-bcd2-e0e4795ab562 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:04.759217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591128d-54fd-4be2-bcd2-e0e4795ab562 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:04.759217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591128d-54fd-4be2-bcd2-e0e4795ab562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:04.792043 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d591128d-54fd-4be2-bcd2-e0e4795ab562 None None] MFA Rules not processed for user `1c8fa181da604f8cb41837dd570f9510`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:37:04.793277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d591128d-54fd-4be2-bcd2-e0e4795ab562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:04.869906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1182/2379] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:37:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:37:04.882377 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd1406-fb41-4df2-bb70-6f01bb657806 service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:04.883909 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-40dd1406-fb41-4df2-bb70-6f01bb657806 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:04.969876 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd1406-fb41-4df2-bb70-6f01bb657806 service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:04.973572 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec519f-a693-4885-902d-57b464de6e79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:04.974237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec519f-a693-4885-902d-57b464de6e79 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:04.974526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec519f-a693-4885-902d-57b464de6e79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:04.974808 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec519f-a693-4885-902d-57b464de6e79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:04.984338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec519f-a693-4885-902d-57b464de6e79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:04.985733 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec519f-a693-4885-902d-57b464de6e79 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:04.998395 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1198/2380] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:37:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:37:06.530297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b6cdd-bf4a-45b8-89e4-0103d7afa9ac None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:06.530853 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b6cdd-bf4a-45b8-89e4-0103d7afa9ac None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:06.530853 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b6cdd-bf4a-45b8-89e4-0103d7afa9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:06.551010 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f10b6cdd-bf4a-45b8-89e4-0103d7afa9ac None None] MFA Rules not processed for user `1c8fa181da604f8cb41837dd570f9510`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:37:06.552282 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f10b6cdd-bf4a-45b8-89e4-0103d7afa9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:06.564747 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1183/2381] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:37:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:37:07.907939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686f20b-3060-49e6-a958-8379d97fa3ef None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:07.908794 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686f20b-3060-49e6-a958-8379d97fa3ef None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:07.909423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686f20b-3060-49e6-a958-8379d97fa3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:07.929989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-9686f20b-3060-49e6-a958-8379d97fa3ef None None] MFA Rules not processed for user `4f75b8b484384554ad59aa20535875e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:37:07.931819 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9686f20b-3060-49e6-a958-8379d97fa3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:08.052780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1199/2382] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 14 20:37:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:37:08.066191 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b6cdd-bf4a-45b8-89e4-0103d7afa9ac None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:08.068678 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f10b6cdd-bf4a-45b8-89e4-0103d7afa9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:08.221246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b6cdd-bf4a-45b8-89e4-0103d7afa9ac None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:08.234876 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf1517-b4b0-4d3a-ab53-6d59f155276a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc23233929144a5b7b3a1733da19828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:08.235716 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf1517-b4b0-4d3a-ab53-6d59f155276a service swift] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:08.236211 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf1517-b4b0-4d3a-ab53-6d59f155276a service swift] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:08.236632 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf1517-b4b0-4d3a-ab53-6d59f155276a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:08.248528 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf1517-b4b0-4d3a-ab53-6d59f155276a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:08.252546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf1517-b4b0-4d3a-ab53-6d59f155276a service swift] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:08.262410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1184/2383] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 14 20:37:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 20:37:08.455743 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a77bb-04d0-4728-8aa8-2f77e145faee None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:08.456576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a77bb-04d0-4728-8aa8-2f77e145faee None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:08.457057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a77bb-04d0-4728-8aa8-2f77e145faee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:08.506885 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-2d0a77bb-04d0-4728-8aa8-2f77e145faee None None] MFA Rules not processed for user `6502e3ffa3b84333a00ea5f712767d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:37:08.512625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-2d0a77bb-04d0-4728-8aa8-2f77e145faee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:08.609139 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:37:08.723763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1200/2384] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:37:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:37:18.258189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:37:18.694550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=13 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:37:29.793757 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55cdfc-f57a-477d-8730-d3b44e1a4e79 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:29.794286 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55cdfc-f57a-477d-8730-d3b44e1a4e79 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:29.794775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55cdfc-f57a-477d-8730-d3b44e1a4e79 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:29.797254 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1185/2385] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 14 20:37:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 20:37:29.806608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d503ff-aaa8-4a10-9b55-c76b5bc4bf73 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:29.807718 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d503ff-aaa8-4a10-9b55-c76b5bc4bf73 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:29.807718 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d503ff-aaa8-4a10-9b55-c76b5bc4bf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:29.846524 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-59d503ff-aaa8-4a10-9b55-c76b5bc4bf73 None None] MFA Rules not processed for user `aa3bb9f1e24a4e9ebd460e97b3736bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:37:29.848069 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-59d503ff-aaa8-4a10-9b55-c76b5bc4bf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:29.947732 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1201/2386] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 14 20:37:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:37:29.960496 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55cdfc-f57a-477d-8730-d3b44e1a4e79 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:29.962338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-df55cdfc-f57a-477d-8730-d3b44e1a4e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:30.028405 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55cdfc-f57a-477d-8730-d3b44e1a4e79 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:30.032484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ca930-4c41-4947-9533-b5aa7c59309e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b7a38b4a7754037a00d11dafc88645b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:30.033128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ca930-4c41-4947-9533-b5aa7c59309e service glance] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:30.033410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ca930-4c41-4947-9533-b5aa7c59309e service glance] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:30.033685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ca930-4c41-4947-9533-b5aa7c59309e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:30.044012 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ca930-4c41-4947-9533-b5aa7c59309e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:30.045471 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ca930-4c41-4947-9533-b5aa7c59309e service glance] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:30.056721 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1186/2387] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:37:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:37:30.122588 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f0a59-97b3-4fca-97f4-e6ceb382c5a6 None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:30.122877 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f0a59-97b3-4fca-97f4-e6ceb382c5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:30.123191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f0a59-97b3-4fca-97f4-e6ceb382c5a6 None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:30.124591 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1202/2388] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 14 20:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:37:30.129530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a486d-55cd-458e-b7b2-debc99d8ae3f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:30.129914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a486d-55cd-458e-b7b2-debc99d8ae3f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:30.130252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a486d-55cd-458e-b7b2-debc99d8ae3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:30.148914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4e4a486d-55cd-458e-b7b2-debc99d8ae3f None None] MFA Rules not processed for user `f88257850ee34ad895208ab935b6e643`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:37:30.150098 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4e4a486d-55cd-458e-b7b2-debc99d8ae3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:30.157109 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1187/2389] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 14 20:37:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:37:30.169816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f0a59-97b3-4fca-97f4-e6ceb382c5a6 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:30.172073 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e86f0a59-97b3-4fca-97f4-e6ceb382c5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:30.191077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f0a59-97b3-4fca-97f4-e6ceb382c5a6 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:30.194770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2c435-f501-4761-88bb-371c4dff8885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:30.195514 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2c435-f501-4761-88bb-371c4dff8885 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:30.195861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2c435-f501-4761-88bb-371c4dff8885 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:30.196268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2c435-f501-4761-88bb-371c4dff8885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:30.204076 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2c435-f501-4761-88bb-371c4dff8885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:30.207898 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2c435-f501-4761-88bb-371c4dff8885 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:30.216208 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1203/2390] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:37:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:37:30.223965 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea8750-c2e0-43ff-b84a-50c1a600f0d6 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:30.224263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea8750-c2e0-43ff-b84a-50c1a600f0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:30.224545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea8750-c2e0-43ff-b84a-50c1a600f0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:30.245226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-30ea8750-c2e0-43ff-b84a-50c1a600f0d6 None None] MFA Rules not processed for user `1c8fa181da604f8cb41837dd570f9510`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:37:30.246591 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-30ea8750-c2e0-43ff-b84a-50c1a600f0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:30.260380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1188/2391] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:37:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:37:30.319227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f2697-22e5-4ab8-9473-9903a89c153f None None] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:30.319537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f2697-22e5-4ab8-9473-9903a89c153f None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:30.319948 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f2697-22e5-4ab8-9473-9903a89c153f None None] PATH_INFO: `/` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:30.321554 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1204/2392] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 14 20:37:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:37:30.327570 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2acee-46a1-460d-9ce6-2c287f5a7348 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:30.327917 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2acee-46a1-460d-9ce6-2c287f5a7348 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:30.328189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2acee-46a1-460d-9ce6-2c287f5a7348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:30.348059 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-5be2acee-46a1-460d-9ce6-2c287f5a7348 None None] MFA Rules not processed for user `aa3bb9f1e24a4e9ebd460e97b3736bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:37:30.349267 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5be2acee-46a1-460d-9ce6-2c287f5a7348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:30.360074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1189/2393] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 14 20:37:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:37:30.371825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f2697-22e5-4ab8-9473-9903a89c153f None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:30.392566 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f2697-22e5-4ab8-9473-9903a89c153f None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:30.397084 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bed535-2c15-445f-96f7-e48204af6e8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:30.398005 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bed535-2c15-445f-96f7-e48204af6e8e service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:30.398483 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bed535-2c15-445f-96f7-e48204af6e8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:30.398973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bed535-2c15-445f-96f7-e48204af6e8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:30.412834 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bed535-2c15-445f-96f7-e48204af6e8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:30.415520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bed535-2c15-445f-96f7-e48204af6e8e service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:30.429450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1205/2394] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:37:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:37:32.474708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2acee-46a1-460d-9ce6-2c287f5a7348 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:32.484203 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2acee-46a1-460d-9ce6-2c287f5a7348 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:32.488449 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308875c-1878-4c7f-b3c8-77b5e8588218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:32.489097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308875c-1878-4c7f-b3c8-77b5e8588218 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:32.489383 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308875c-1878-4c7f-b3c8-77b5e8588218 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:32.489661 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308875c-1878-4c7f-b3c8-77b5e8588218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:32.492207 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d308875c-1878-4c7f-b3c8-77b5e8588218 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:32.561438 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308875c-1878-4c7f-b3c8-77b5e8588218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:32.562904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308875c-1878-4c7f-b3c8-77b5e8588218 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:32.572879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1190/2395] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:37:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:37:37.247727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba786dc5-4f20-4cf4-becb-a4148d1783aa None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:37.248279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba786dc5-4f20-4cf4-becb-a4148d1783aa None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:37.248279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba786dc5-4f20-4cf4-becb-a4148d1783aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:37.272968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-ba786dc5-4f20-4cf4-becb-a4148d1783aa None None] MFA Rules not processed for user `4f75b8b484384554ad59aa20535875e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:37:37.274255 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ba786dc5-4f20-4cf4-becb-a4148d1783aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:37.282635 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1206/2396] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 14 20:37:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:37:37.297332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308875c-1878-4c7f-b3c8-77b5e8588218 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:37.310316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308875c-1878-4c7f-b3c8-77b5e8588218 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:37.316062 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7698e-1737-4894-a2cf-5ae91b159980 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc23233929144a5b7b3a1733da19828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:37.316715 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7698e-1737-4894-a2cf-5ae91b159980 service swift] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:37.317097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7698e-1737-4894-a2cf-5ae91b159980 service swift] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:37.317279 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7698e-1737-4894-a2cf-5ae91b159980 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:37.325762 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba7698e-1737-4894-a2cf-5ae91b159980 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:37.327420 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba7698e-1737-4894-a2cf-5ae91b159980 service swift] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:37.335753 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1191/2397] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 14 20:37:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 20:37:47.266142 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=46 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:37:47.332324 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=71 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:37:54.911002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba786dc5-4f20-4cf4-becb-a4148d1783aa None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:37:54.912443 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ba786dc5-4f20-4cf4-becb-a4148d1783aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:54.980218 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba786dc5-4f20-4cf4-becb-a4148d1783aa None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:37:54.985160 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a1d1f-c2e2-4931-8727-02334d9ddb03 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afad68819504c2b864851c6989ae25a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:37:54.985803 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a1d1f-c2e2-4931-8727-02334d9ddb03 service placement] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:37:54.986083 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a1d1f-c2e2-4931-8727-02334d9ddb03 service placement] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:37:54.986399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a1d1f-c2e2-4931-8727-02334d9ddb03 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:37:54.988664 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-094a1d1f-c2e2-4931-8727-02334d9ddb03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:37:55.000235 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a1d1f-c2e2-4931-8727-02334d9ddb03 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:37:55.001852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a1d1f-c2e2-4931-8727-02334d9ddb03 service placement] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:37:55.011714 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1207/2398] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:37:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:38:00.619956 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031559a6-4797-474a-a4be-858a6a054cd9 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:00.620453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031559a6-4797-474a-a4be-858a6a054cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:00.620552 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031559a6-4797-474a-a4be-858a6a054cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:00.639559 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-031559a6-4797-474a-a4be-858a6a054cd9 None None] MFA Rules not processed for user `4f75b8b484384554ad59aa20535875e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:38:00.640760 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-031559a6-4797-474a-a4be-858a6a054cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:00.648970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1192/2399] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 14 20:38:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:38:00.671347 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a1d1f-c2e2-4931-8727-02334d9ddb03 service placement] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:00.682839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a1d1f-c2e2-4931-8727-02334d9ddb03 service placement] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:00.687788 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be474fc5-5f82-44ea-8807-ffdff3d87a2b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc23233929144a5b7b3a1733da19828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:00.688461 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be474fc5-5f82-44ea-8807-ffdff3d87a2b service swift] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:00.688747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be474fc5-5f82-44ea-8807-ffdff3d87a2b service swift] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:00.689025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be474fc5-5f82-44ea-8807-ffdff3d87a2b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:00.697924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be474fc5-5f82-44ea-8807-ffdff3d87a2b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:00.699618 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be474fc5-5f82-44ea-8807-ffdff3d87a2b service swift] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:00.708734 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1208/2400] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 14 20:38:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 20:38:04.445351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64ba70-fc12-4710-afff-3770848bd450 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:04.446334 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64ba70-fc12-4710-afff-3770848bd450 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:04.446806 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64ba70-fc12-4710-afff-3770848bd450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:04.512680 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-6a64ba70-fc12-4710-afff-3770848bd450 None None] MFA Rules not processed for user `4f75b8b484384554ad59aa20535875e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:38:04.523039 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6a64ba70-fc12-4710-afff-3770848bd450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:04.538431 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1193/2401] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jul 14 20:38:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:38:04.567667 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be474fc5-5f82-44ea-8807-ffdff3d87a2b service swift] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:04.585363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be474fc5-5f82-44ea-8807-ffdff3d87a2b service swift] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:04.600565 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87800a2-e4bf-4bda-98e7-6a74064b4c29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc23233929144a5b7b3a1733da19828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:04.601493 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87800a2-e4bf-4bda-98e7-6a74064b4c29 service swift] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:04.601952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87800a2-e4bf-4bda-98e7-6a74064b4c29 service swift] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:04.602528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87800a2-e4bf-4bda-98e7-6a74064b4c29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:04.618180 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87800a2-e4bf-4bda-98e7-6a74064b4c29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:04.620374 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87800a2-e4bf-4bda-98e7-6a74064b4c29 service swift] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:04.635147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1209/2402] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 14 20:38:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 20:38:14.493459 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=6 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:38:14.629552 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=44 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:38:18.915062 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dcc43-917a-4cc8-bc2e-e28547998c4e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:18.915632 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dcc43-917a-4cc8-bc2e-e28547998c4e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:18.915632 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dcc43-917a-4cc8-bc2e-e28547998c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:18.935117 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8d1dcc43-917a-4cc8-bc2e-e28547998c4e None None] MFA Rules not processed for user `4f75b8b484384554ad59aa20535875e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:38:18.936371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8d1dcc43-917a-4cc8-bc2e-e28547998c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:18.948944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1194/2403] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jul 14 20:38:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:38:18.956903 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87800a2-e4bf-4bda-98e7-6a74064b4c29 service swift] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:18.968532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87800a2-e4bf-4bda-98e7-6a74064b4c29 service swift] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:18.974580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb767ed-e39d-4969-8201-3d36a56ad5e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc23233929144a5b7b3a1733da19828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:18.975407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb767ed-e39d-4969-8201-3d36a56ad5e5 service swift] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:18.975582 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb767ed-e39d-4969-8201-3d36a56ad5e5 service swift] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:18.975872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb767ed-e39d-4969-8201-3d36a56ad5e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:18.986607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb767ed-e39d-4969-8201-3d36a56ad5e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:18.988246 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb767ed-e39d-4969-8201-3d36a56ad5e5 service swift] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:18.997084 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1210/2404] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 14 20:38:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 20:38:21.472047 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dcc43-917a-4cc8-bc2e-e28547998c4e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:21.485887 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dcc43-917a-4cc8-bc2e-e28547998c4e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:21.490829 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d9ff4-d0bc-43d8-9342-5df4a86bd98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:21.491777 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d9ff4-d0bc-43d8-9342-5df4a86bd98a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:21.492054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d9ff4-d0bc-43d8-9342-5df4a86bd98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:21.492528 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d9ff4-d0bc-43d8-9342-5df4a86bd98a admin admin] PATH_INFO: `/v3/users/1c8fa181da604f8cb41837dd570f9510` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:21.493902 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451d9ff4-d0bc-43d8-9342-5df4a86bd98a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c8fa181da604f8cb41837dd570f9510)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:21.495883 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451d9ff4-d0bc-43d8-9342-5df4a86bd98a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:21.557116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-451d9ff4-d0bc-43d8-9342-5df4a86bd98a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c8fa181da604f8cb41837dd570f9510'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:21.566821 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-451d9ff4-d0bc-43d8-9342-5df4a86bd98a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c8fa181da604f8cb41837dd570f9510'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:21.567793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-451d9ff4-d0bc-43d8-9342-5df4a86bd98a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c8fa181da604f8cb41837dd570f9510'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:21.568416 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-451d9ff4-d0bc-43d8-9342-5df4a86bd98a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c8fa181da604f8cb41837dd570f9510'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:21.583585 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-451d9ff4-d0bc-43d8-9342-5df4a86bd98a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c8fa181da604f8cb41837dd570f9510'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:21.604260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1195/2405] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:38:21 2022] DELETE /identity/v3/users/1c8fa181da604f8cb41837dd570f9510 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:38:21.792093 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb767ed-e39d-4969-8201-3d36a56ad5e5 service swift] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:21.803909 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb767ed-e39d-4969-8201-3d36a56ad5e5 service swift] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:21.899269 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5faa8-53ea-4d6f-ac1f-018dd91bc0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:21.900181 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5faa8-53ea-4d6f-ac1f-018dd91bc0a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:21.900469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5faa8-53ea-4d6f-ac1f-018dd91bc0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:21.900747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5faa8-53ea-4d6f-ac1f-018dd91bc0a0 admin admin] PATH_INFO: `/v3/projects/806c059a19044c6ebcd8f2cc50d9591d` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:21.901990 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5faa8-53ea-4d6f-ac1f-018dd91bc0a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=806c059a19044c6ebcd8f2cc50d9591d)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:21.904136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5faa8-53ea-4d6f-ac1f-018dd91bc0a0 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:21.942456 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4886a4b-03dd-464b-a481-36b07d47f524 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:21.943051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4886a4b-03dd-464b-a481-36b07d47f524 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:21.943474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4886a4b-03dd-464b-a481-36b07d47f524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:21.957677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-96f5faa8-53ea-4d6f-ac1f-018dd91bc0a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '806c059a19044c6ebcd8f2cc50d9591d', 'request_id': 'req-96f5faa8-53ea-4d6f-ac1f-018dd91bc0a0'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:21.963398 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c4886a4b-03dd-464b-a481-36b07d47f524 None None] MFA Rules not processed for user `16c6975c73954487a1f15649693b1d8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:38:21.964619 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c4886a4b-03dd-464b-a481-36b07d47f524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:21.967595 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-96f5faa8-53ea-4d6f-ac1f-018dd91bc0a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '806c059a19044c6ebcd8f2cc50d9591d', 'request_id': 'req-96f5faa8-53ea-4d6f-ac1f-018dd91bc0a0'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:21.974206 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-96f5faa8-53ea-4d6f-ac1f-018dd91bc0a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '806c059a19044c6ebcd8f2cc50d9591d', 'request_id': 'req-96f5faa8-53ea-4d6f-ac1f-018dd91bc0a0'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:21.985149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-96f5faa8-53ea-4d6f-ac1f-018dd91bc0a0 admin admin] The token cache is being invalidate because project 806c059a19044c6ebcd8f2cc50d9591d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:38:21.985501 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-96f5faa8-53ea-4d6f-ac1f-018dd91bc0a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:21.986212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-96f5faa8-53ea-4d6f-ac1f-018dd91bc0a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:21.988229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1211/2406] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:38:21 2022] DELETE /identity/v3/projects/806c059a19044c6ebcd8f2cc50d9591d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:38:21.992707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5b41c-5bb8-4eae-8903-d3a5ed0462f9 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:21.992987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5b41c-5bb8-4eae-8903-d3a5ed0462f9 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:21.993577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5b41c-5bb8-4eae-8903-d3a5ed0462f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:22.013470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-7ff5b41c-5bb8-4eae-8903-d3a5ed0462f9 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:38:22.014918 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-7ff5b41c-5bb8-4eae-8903-d3a5ed0462f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:22.027390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1196/2407] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:38:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:38:22.091476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1212/2408] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:38:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:38:22.099229 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566d583-1a39-4aa5-9aa0-b8f1e0820cb8 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:22.099526 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566d583-1a39-4aa5-9aa0-b8f1e0820cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:22.099848 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566d583-1a39-4aa5-9aa0-b8f1e0820cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:22.118439 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-6566d583-1a39-4aa5-9aa0-b8f1e0820cb8 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:38:22.119713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6566d583-1a39-4aa5-9aa0-b8f1e0820cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:22.128262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1197/2409] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:38:22.132490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5b41c-5bb8-4eae-8903-d3a5ed0462f9 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:22.142469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5b41c-5bb8-4eae-8903-d3a5ed0462f9 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:22.147686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82d548-702f-4f0c-8740-287bdc75e884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:22.148357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82d548-702f-4f0c-8740-287bdc75e884 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:22.148637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82d548-702f-4f0c-8740-287bdc75e884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:22.148915 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82d548-702f-4f0c-8740-287bdc75e884 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:22.149649 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d82d548-702f-4f0c-8740-287bdc75e884 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:38:22.149955 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d82d548-702f-4f0c-8740-287bdc75e884 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:22.151572 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d82d548-702f-4f0c-8740-287bdc75e884 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:22.168382 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1213/2410] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:38:22.170905 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566d583-1a39-4aa5-9aa0-b8f1e0820cb8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:22.183636 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566d583-1a39-4aa5-9aa0-b8f1e0820cb8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:22.190228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca595aa-31a8-465d-aab6-557dd14fbb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:22.190900 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca595aa-31a8-465d-aab6-557dd14fbb78 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:22.191217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca595aa-31a8-465d-aab6-557dd14fbb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:22.191496 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca595aa-31a8-465d-aab6-557dd14fbb78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:22.192208 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca595aa-31a8-465d-aab6-557dd14fbb78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:22.194123 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca595aa-31a8-465d-aab6-557dd14fbb78 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:22.223114 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1198/2411] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:38:22 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:38:22.227203 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82d548-702f-4f0c-8740-287bdc75e884 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:22.235886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:38:22.239802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82d548-702f-4f0c-8740-287bdc75e884 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:22.309900 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05eb1e-b6a4-47e7-a156-a27efde57f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:22.310589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05eb1e-b6a4-47e7-a156-a27efde57f00 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:22.310961 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05eb1e-b6a4-47e7-a156-a27efde57f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:22.311314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05eb1e-b6a4-47e7-a156-a27efde57f00 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:22.312124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b05eb1e-b6a4-47e7-a156-a27efde57f00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:22.314121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b05eb1e-b6a4-47e7-a156-a27efde57f00 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:22.336446 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1214/2412] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:38:22 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:38:22.340806 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca595aa-31a8-465d-aab6-557dd14fbb78 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:22.352039 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca595aa-31a8-465d-aab6-557dd14fbb78 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:22.356904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055442b-934e-4937-96c3-09d96c1a8f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:22.357662 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055442b-934e-4937-96c3-09d96c1a8f64 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:22.357947 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055442b-934e-4937-96c3-09d96c1a8f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:22.358223 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055442b-934e-4937-96c3-09d96c1a8f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:22.359019 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055442b-934e-4937-96c3-09d96c1a8f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:22.360726 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055442b-934e-4937-96c3-09d96c1a8f64 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:22.371734 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1199/2413] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:38:22.375855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05eb1e-b6a4-47e7-a156-a27efde57f00 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:22.386016 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05eb1e-b6a4-47e7-a156-a27efde57f00 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:22.390655 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfd91f-fb34-4759-81a9-fb469fb75dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:22.391350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfd91f-fb34-4759-81a9-fb469fb75dba admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:22.392153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfd91f-fb34-4759-81a9-fb469fb75dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:22.392430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfd91f-fb34-4759-81a9-fb469fb75dba admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:22.393162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfd91f-fb34-4759-81a9-fb469fb75dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:22.394749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfd91f-fb34-4759-81a9-fb469fb75dba admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:22.405130 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1215/2414] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:38:22.409957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055442b-934e-4937-96c3-09d96c1a8f64 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:22.420826 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055442b-934e-4937-96c3-09d96c1a8f64 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:22.425790 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c0370-b267-4a26-b804-edaecc70970e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:22.426687 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c0370-b267-4a26-b804-edaecc70970e admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:22.427163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c0370-b267-4a26-b804-edaecc70970e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:22.427445 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c0370-b267-4a26-b804-edaecc70970e admin admin] PATH_INFO: `/v3/projects/b10c07b356db4ad29541868b1f7af207/users/71f9dcead66c41d988ae745792ee39d6/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:22.443029 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c0370-b267-4a26-b804-edaecc70970e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10c07b356db4ad29541868b1f7af207, user_id=71f9dcead66c41d988ae745792ee39d6, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:22.445699 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c0370-b267-4a26-b804-edaecc70970e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:22.460973 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1200/2415] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:38:22 2022] PUT /identity/v3/projects/b10c07b356db4ad29541868b1f7af207/users/71f9dcead66c41d988ae745792ee39d6/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:38:22.528186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b10eba-762d-4ead-a4b3-00dd29aa20b4 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:22.528475 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b10eba-762d-4ead-a4b3-00dd29aa20b4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:22.528753 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b10eba-762d-4ead-a4b3-00dd29aa20b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:22.561429 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-c5b10eba-762d-4ead-a4b3-00dd29aa20b4 None None] MFA Rules not processed for user `71f9dcead66c41d988ae745792ee39d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:38:22.562896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-c5b10eba-762d-4ead-a4b3-00dd29aa20b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:22.652440 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1216/2416] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:38:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:38:22.663261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c0370-b267-4a26-b804-edaecc70970e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:22.665590 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-866c0370-b267-4a26-b804-edaecc70970e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:22.676288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:38:22.730119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c0370-b267-4a26-b804-edaecc70970e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:22.734501 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69459af6-dcc4-4898-8962-4c8c4aa8249c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:22.736275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69459af6-dcc4-4898-8962-4c8c4aa8249c service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:22.736275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69459af6-dcc4-4898-8962-4c8c4aa8249c service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:22.737174 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69459af6-dcc4-4898-8962-4c8c4aa8249c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:22.747183 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69459af6-dcc4-4898-8962-4c8c4aa8249c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:22.748669 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69459af6-dcc4-4898-8962-4c8c4aa8249c service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:22.759499 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1201/2417] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:38:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:38:32.641408 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=64,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:38:32.754424 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=23 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:38:33.129070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b10eba-762d-4ead-a4b3-00dd29aa20b4 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:33.189293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b10eba-762d-4ead-a4b3-00dd29aa20b4 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:33.193676 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33737d-0957-4e7a-b1b5-0ea3960ac482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:33.194545 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33737d-0957-4e7a-b1b5-0ea3960ac482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:33.194823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33737d-0957-4e7a-b1b5-0ea3960ac482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:33.195127 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33737d-0957-4e7a-b1b5-0ea3960ac482 admin admin] PATH_INFO: `/v3/users/71f9dcead66c41d988ae745792ee39d6` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:33.196308 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf33737d-0957-4e7a-b1b5-0ea3960ac482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71f9dcead66c41d988ae745792ee39d6)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:33.198228 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf33737d-0957-4e7a-b1b5-0ea3960ac482 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:33.245110 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cf33737d-0957-4e7a-b1b5-0ea3960ac482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71f9dcead66c41d988ae745792ee39d6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:33.253385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cf33737d-0957-4e7a-b1b5-0ea3960ac482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71f9dcead66c41d988ae745792ee39d6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:33.260358 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cf33737d-0957-4e7a-b1b5-0ea3960ac482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71f9dcead66c41d988ae745792ee39d6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:33.260993 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cf33737d-0957-4e7a-b1b5-0ea3960ac482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71f9dcead66c41d988ae745792ee39d6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:33.261594 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-cf33737d-0957-4e7a-b1b5-0ea3960ac482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71f9dcead66c41d988ae745792ee39d6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:33.279170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1217/2418] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:38:33 2022] DELETE /identity/v3/users/71f9dcead66c41d988ae745792ee39d6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:38:33.284230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995caf3-ced1-4964-b50b-a886a434ebe8 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:33.284519 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995caf3-ced1-4964-b50b-a886a434ebe8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:33.284824 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995caf3-ced1-4964-b50b-a886a434ebe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:33.303361 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1995caf3-ced1-4964-b50b-a886a434ebe8 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:38:33.304555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1995caf3-ced1-4964-b50b-a886a434ebe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:33.361321 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1202/2419] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:38:33.371629 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33737d-0957-4e7a-b1b5-0ea3960ac482 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:33.373029 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-cf33737d-0957-4e7a-b1b5-0ea3960ac482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:33.432378 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33737d-0957-4e7a-b1b5-0ea3960ac482 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:33.436322 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5d6b9-1be4-49ae-bac4-bb85e4bb7fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:33.436972 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5d6b9-1be4-49ae-bac4-bb85e4bb7fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:33.437277 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5d6b9-1be4-49ae-bac4-bb85e4bb7fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:33.437591 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5d6b9-1be4-49ae-bac4-bb85e4bb7fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:33.448670 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5d6b9-1be4-49ae-bac4-bb85e4bb7fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:33.450140 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5d6b9-1be4-49ae-bac4-bb85e4bb7fb1 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:33.461358 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1218/2420] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:38:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:38:33.733680 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995caf3-ced1-4964-b50b-a886a434ebe8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:33.736034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1995caf3-ced1-4964-b50b-a886a434ebe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:33.749026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995caf3-ced1-4964-b50b-a886a434ebe8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:33.754959 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaaf639-786b-4783-ab8a-8bceee7c1be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:33.756092 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaf639-786b-4783-ab8a-8bceee7c1be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:33.756449 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaf639-786b-4783-ab8a-8bceee7c1be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:33.756800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaf639-786b-4783-ab8a-8bceee7c1be2 admin admin] PATH_INFO: `/v3/projects/b10c07b356db4ad29541868b1f7af207` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:33.758102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaaf639-786b-4783-ab8a-8bceee7c1be2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10c07b356db4ad29541868b1f7af207)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:33.760145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaaf639-786b-4783-ab8a-8bceee7c1be2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:33.809717 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-caaaf639-786b-4783-ab8a-8bceee7c1be2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10c07b356db4ad29541868b1f7af207', 'request_id': 'req-caaaf639-786b-4783-ab8a-8bceee7c1be2'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:33.816113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-caaaf639-786b-4783-ab8a-8bceee7c1be2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10c07b356db4ad29541868b1f7af207', 'request_id': 'req-caaaf639-786b-4783-ab8a-8bceee7c1be2'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:33.821621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-caaaf639-786b-4783-ab8a-8bceee7c1be2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10c07b356db4ad29541868b1f7af207', 'request_id': 'req-caaaf639-786b-4783-ab8a-8bceee7c1be2'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:33.835366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-caaaf639-786b-4783-ab8a-8bceee7c1be2 admin admin] The token cache is being invalidate because project b10c07b356db4ad29541868b1f7af207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:38:33.835657 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-caaaf639-786b-4783-ab8a-8bceee7c1be2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:33.836282 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-caaaf639-786b-4783-ab8a-8bceee7c1be2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:33.837819 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1203/2421] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:38:33 2022] DELETE /identity/v3/projects/b10c07b356db4ad29541868b1f7af207 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:38:33.842916 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c90f7-f92c-43d1-b611-e58173f2b0f6 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:33.843407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c90f7-f92c-43d1-b611-e58173f2b0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:33.843778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c90f7-f92c-43d1-b611-e58173f2b0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:33.865333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-022c90f7-f92c-43d1-b611-e58173f2b0f6 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:38:33.866619 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-022c90f7-f92c-43d1-b611-e58173f2b0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:33.886438 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:38:33.935087 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1219/2422] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:38:33.943531 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc13a72-5c4a-43f9-8af9-8fbf3f7d99bf None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:33.943844 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc13a72-5c4a-43f9-8af9-8fbf3f7d99bf None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:33.944123 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc13a72-5c4a-43f9-8af9-8fbf3f7d99bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:33.963558 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-dfc13a72-5c4a-43f9-8af9-8fbf3f7d99bf None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:38:33.964777 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dfc13a72-5c4a-43f9-8af9-8fbf3f7d99bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:33.977346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1204/2423] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:38:33.978539 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c90f7-f92c-43d1-b611-e58173f2b0f6 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:33.988984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c90f7-f92c-43d1-b611-e58173f2b0f6 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:33.994097 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247bb51-6ed6-4a80-a7e2-5d6a0520c107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:33.994974 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247bb51-6ed6-4a80-a7e2-5d6a0520c107 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:33.995112 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247bb51-6ed6-4a80-a7e2-5d6a0520c107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:33.995355 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247bb51-6ed6-4a80-a7e2-5d6a0520c107 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:33.996099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247bb51-6ed6-4a80-a7e2-5d6a0520c107 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:38:33.996408 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247bb51-6ed6-4a80-a7e2-5d6a0520c107 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:33.998016 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247bb51-6ed6-4a80-a7e2-5d6a0520c107 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:34.010859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1220/2424] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:38:34.015169 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc13a72-5c4a-43f9-8af9-8fbf3f7d99bf None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:34.025438 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc13a72-5c4a-43f9-8af9-8fbf3f7d99bf None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:34.030297 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207797a6-6d15-4b5c-989d-503fd4822ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:34.030987 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207797a6-6d15-4b5c-989d-503fd4822ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:34.031744 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207797a6-6d15-4b5c-989d-503fd4822ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:34.031744 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207797a6-6d15-4b5c-989d-503fd4822ed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:34.032248 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207797a6-6d15-4b5c-989d-503fd4822ed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:34.034158 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207797a6-6d15-4b5c-989d-503fd4822ed0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:34.064337 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1205/2425] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:38:34 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:38:34.068704 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247bb51-6ed6-4a80-a7e2-5d6a0520c107 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:34.077495 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ba28d0-e1b5-4dac-b678-e59f297007d9 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:34.077797 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ba28d0-e1b5-4dac-b678-e59f297007d9 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:34.078066 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ba28d0-e1b5-4dac-b678-e59f297007d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:34.090578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247bb51-6ed6-4a80-a7e2-5d6a0520c107 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:34.096762 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-04ba28d0-e1b5-4dac-b678-e59f297007d9 None None] MFA Rules not processed for user `765c911957bd4ca68dd520fc4fbba1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:38:34.097977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-04ba28d0-e1b5-4dac-b678-e59f297007d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:34.146881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:38:34.154526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d2294-1753-4b70-be1f-3e49475ff96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:34.155327 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d2294-1753-4b70-be1f-3e49475ff96e admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:34.155549 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d2294-1753-4b70-be1f-3e49475ff96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:34.155826 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d2294-1753-4b70-be1f-3e49475ff96e admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:34.156644 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d2294-1753-4b70-be1f-3e49475ff96e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:34.157605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1206/2426] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:38:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:38:34.158551 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d2294-1753-4b70-be1f-3e49475ff96e admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:34.182625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1221/2427] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:38:34 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:38:34.184487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ba28d0-e1b5-4dac-b678-e59f297007d9 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:34.195955 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ba28d0-e1b5-4dac-b678-e59f297007d9 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:34.200536 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d1f85-1a5e-4800-afe1-23a1e6cf95a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:34.201191 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d1f85-1a5e-4800-afe1-23a1e6cf95a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:34.201470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d1f85-1a5e-4800-afe1-23a1e6cf95a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:34.201746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d1f85-1a5e-4800-afe1-23a1e6cf95a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:34.202465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d1f85-1a5e-4800-afe1-23a1e6cf95a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:34.204178 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d1f85-1a5e-4800-afe1-23a1e6cf95a9 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:34.214460 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1207/2428] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:38:34.218865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d2294-1753-4b70-be1f-3e49475ff96e admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:34.229278 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d2294-1753-4b70-be1f-3e49475ff96e admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:34.234386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858887f-5e38-4ea3-a828-cf9cf0155c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:34.235666 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858887f-5e38-4ea3-a828-cf9cf0155c74 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:34.236039 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858887f-5e38-4ea3-a828-cf9cf0155c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:34.236383 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858887f-5e38-4ea3-a828-cf9cf0155c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:34.237183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858887f-5e38-4ea3-a828-cf9cf0155c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:34.238841 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858887f-5e38-4ea3-a828-cf9cf0155c74 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:34.253001 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1222/2429] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:38:34.253901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d1f85-1a5e-4800-afe1-23a1e6cf95a9 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:34.265144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d1f85-1a5e-4800-afe1-23a1e6cf95a9 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:34.269837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb0130-9f58-43a2-be34-4f71d1492eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:34.270995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb0130-9f58-43a2-be34-4f71d1492eb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:34.271277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb0130-9f58-43a2-be34-4f71d1492eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:34.271547 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb0130-9f58-43a2-be34-4f71d1492eb4 admin admin] PATH_INFO: `/v3/projects/74bc9fe902c9428086839a9c5a532e95/users/55468cc8f02e452981e80e7ae655b2b4/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:34.288991 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb0130-9f58-43a2-be34-4f71d1492eb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74bc9fe902c9428086839a9c5a532e95, user_id=55468cc8f02e452981e80e7ae655b2b4, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:34.291609 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb0130-9f58-43a2-be34-4f71d1492eb4 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:34.306259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1208/2430] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:38:34 2022] PUT /identity/v3/projects/74bc9fe902c9428086839a9c5a532e95/users/55468cc8f02e452981e80e7ae655b2b4/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:38:34.319046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc8407-37e7-4f2a-9a03-bcd5c9ef3e46 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:34.319046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc8407-37e7-4f2a-9a03-bcd5c9ef3e46 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:34.319046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc8407-37e7-4f2a-9a03-bcd5c9ef3e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:34.347363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-81cc8407-37e7-4f2a-9a03-bcd5c9ef3e46 None None] MFA Rules not processed for user `55468cc8f02e452981e80e7ae655b2b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:38:34.349300 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-81cc8407-37e7-4f2a-9a03-bcd5c9ef3e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:34.429536 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1223/2431] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:38:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:38:34.441204 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb0130-9f58-43a2-be34-4f71d1492eb4 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:34.443386 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ffcb0130-9f58-43a2-be34-4f71d1492eb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:34.532686 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb0130-9f58-43a2-be34-4f71d1492eb4 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:34.536473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c8885-3284-497b-bd20-b6559dd87a41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:34.537110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c8885-3284-497b-bd20-b6559dd87a41 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:34.537392 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c8885-3284-497b-bd20-b6559dd87a41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:34.537665 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c8885-3284-497b-bd20-b6559dd87a41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:34.546312 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c8885-3284-497b-bd20-b6559dd87a41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:34.548271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c8885-3284-497b-bd20-b6559dd87a41 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:34.557571 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1209/2432] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:38:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:38:44.425915 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:38:44.553153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:38:51.791655 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc8407-37e7-4f2a-9a03-bcd5c9ef3e46 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:51.853460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc8407-37e7-4f2a-9a03-bcd5c9ef3e46 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:51.858173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a451be-0a42-4efc-8423-b34b3a760b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:51.859064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a451be-0a42-4efc-8423-b34b3a760b87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:51.859348 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a451be-0a42-4efc-8423-b34b3a760b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:51.859623 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a451be-0a42-4efc-8423-b34b3a760b87 admin admin] PATH_INFO: `/v3/users/55468cc8f02e452981e80e7ae655b2b4` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:51.860819 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a451be-0a42-4efc-8423-b34b3a760b87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55468cc8f02e452981e80e7ae655b2b4)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:51.862748 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a451be-0a42-4efc-8423-b34b3a760b87 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:51.918369 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f7a451be-0a42-4efc-8423-b34b3a760b87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55468cc8f02e452981e80e7ae655b2b4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:51.926076 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f7a451be-0a42-4efc-8423-b34b3a760b87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55468cc8f02e452981e80e7ae655b2b4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:51.932951 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f7a451be-0a42-4efc-8423-b34b3a760b87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55468cc8f02e452981e80e7ae655b2b4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:51.933558 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f7a451be-0a42-4efc-8423-b34b3a760b87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55468cc8f02e452981e80e7ae655b2b4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:51.934197 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-f7a451be-0a42-4efc-8423-b34b3a760b87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55468cc8f02e452981e80e7ae655b2b4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:51.951804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1224/2433] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:38:51 2022] DELETE /identity/v3/users/55468cc8f02e452981e80e7ae655b2b4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:38:51.957093 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706adec5-5b77-41a3-9383-defbc9269fd5 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:51.957587 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706adec5-5b77-41a3-9383-defbc9269fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:51.957686 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706adec5-5b77-41a3-9383-defbc9269fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:51.974632 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-706adec5-5b77-41a3-9383-defbc9269fd5 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:38:51.975855 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-706adec5-5b77-41a3-9383-defbc9269fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:52.034474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1210/2434] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:38:52.044110 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a451be-0a42-4efc-8423-b34b3a760b87 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:52.045532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f7a451be-0a42-4efc-8423-b34b3a760b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:52.102249 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a451be-0a42-4efc-8423-b34b3a760b87 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:52.105912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db47fa5-a6fb-4518-88f9-d0f638e0698f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:52.106554 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db47fa5-a6fb-4518-88f9-d0f638e0698f service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:52.106839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db47fa5-a6fb-4518-88f9-d0f638e0698f service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:52.107145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db47fa5-a6fb-4518-88f9-d0f638e0698f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:52.116545 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db47fa5-a6fb-4518-88f9-d0f638e0698f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:52.117959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db47fa5-a6fb-4518-88f9-d0f638e0698f service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:52.128259 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1225/2435] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:38:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:38:52.406552 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706adec5-5b77-41a3-9383-defbc9269fd5 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:52.408129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-706adec5-5b77-41a3-9383-defbc9269fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:52.420752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706adec5-5b77-41a3-9383-defbc9269fd5 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:52.426218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf69d8-e20b-4b69-a84b-b69d43b644e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:52.427119 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf69d8-e20b-4b69-a84b-b69d43b644e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:52.427408 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf69d8-e20b-4b69-a84b-b69d43b644e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:52.427689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf69d8-e20b-4b69-a84b-b69d43b644e6 admin admin] PATH_INFO: `/v3/projects/74bc9fe902c9428086839a9c5a532e95` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:52.428896 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf69d8-e20b-4b69-a84b-b69d43b644e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74bc9fe902c9428086839a9c5a532e95)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:52.430824 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf69d8-e20b-4b69-a84b-b69d43b644e6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:52.479318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2cdf69d8-e20b-4b69-a84b-b69d43b644e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74bc9fe902c9428086839a9c5a532e95', 'request_id': 'req-2cdf69d8-e20b-4b69-a84b-b69d43b644e6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:52.486483 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2cdf69d8-e20b-4b69-a84b-b69d43b644e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74bc9fe902c9428086839a9c5a532e95', 'request_id': 'req-2cdf69d8-e20b-4b69-a84b-b69d43b644e6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:52.492206 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2cdf69d8-e20b-4b69-a84b-b69d43b644e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74bc9fe902c9428086839a9c5a532e95', 'request_id': 'req-2cdf69d8-e20b-4b69-a84b-b69d43b644e6'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:52.505024 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2cdf69d8-e20b-4b69-a84b-b69d43b644e6 admin admin] The token cache is being invalidate because project 74bc9fe902c9428086839a9c5a532e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:38:52.505315 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2cdf69d8-e20b-4b69-a84b-b69d43b644e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:52.505932 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-2cdf69d8-e20b-4b69-a84b-b69d43b644e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:38:52.507511 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1211/2436] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:38:52 2022] DELETE /identity/v3/projects/74bc9fe902c9428086839a9c5a532e95 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:38:52.512685 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17915be9-3914-4a55-af30-8d6289b44b5e None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:52.513147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17915be9-3914-4a55-af30-8d6289b44b5e None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:52.513268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17915be9-3914-4a55-af30-8d6289b44b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:52.533626 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-17915be9-3914-4a55-af30-8d6289b44b5e None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:38:52.537027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-17915be9-3914-4a55-af30-8d6289b44b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:52.556938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:38:52.602913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1226/2437] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:38:52.611669 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecebcef7-67ee-4803-b5b9-35f0d96bfcc5 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:52.611957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecebcef7-67ee-4803-b5b9-35f0d96bfcc5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:52.612235 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecebcef7-67ee-4803-b5b9-35f0d96bfcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:52.630393 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ecebcef7-67ee-4803-b5b9-35f0d96bfcc5 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:38:52.631656 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ecebcef7-67ee-4803-b5b9-35f0d96bfcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:52.639849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1212/2438] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:38:52.644074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17915be9-3914-4a55-af30-8d6289b44b5e None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:52.653923 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17915be9-3914-4a55-af30-8d6289b44b5e None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:52.659006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf4519-276c-4e42-9dd8-dd73b388d266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:52.659665 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf4519-276c-4e42-9dd8-dd73b388d266 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:52.659948 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf4519-276c-4e42-9dd8-dd73b388d266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:52.660224 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf4519-276c-4e42-9dd8-dd73b388d266 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:52.660951 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf4519-276c-4e42-9dd8-dd73b388d266 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:38:52.661256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf4519-276c-4e42-9dd8-dd73b388d266 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:52.662850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf4519-276c-4e42-9dd8-dd73b388d266 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:52.675439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1227/2439] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:38:52.680118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecebcef7-67ee-4803-b5b9-35f0d96bfcc5 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:52.689407 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecebcef7-67ee-4803-b5b9-35f0d96bfcc5 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:52.694090 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3aa7c6-5d0d-44be-9ec7-4dfa4ab18844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:52.694734 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3aa7c6-5d0d-44be-9ec7-4dfa4ab18844 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:52.695053 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3aa7c6-5d0d-44be-9ec7-4dfa4ab18844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:52.695327 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3aa7c6-5d0d-44be-9ec7-4dfa4ab18844 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:52.696023 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3aa7c6-5d0d-44be-9ec7-4dfa4ab18844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:52.697922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3aa7c6-5d0d-44be-9ec7-4dfa4ab18844 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:52.724003 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1213/2440] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:38:52 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:38:52.727563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf4519-276c-4e42-9dd8-dd73b388d266 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:52.738375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf4519-276c-4e42-9dd8-dd73b388d266 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:52.794887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d1b7d-2580-445b-bf35-1bb334f9b304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:52.795577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d1b7d-2580-445b-bf35-1bb334f9b304 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:52.795871 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d1b7d-2580-445b-bf35-1bb334f9b304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:52.796148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d1b7d-2580-445b-bf35-1bb334f9b304 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:52.796841 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d1b7d-2580-445b-bf35-1bb334f9b304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:52.799435 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d1b7d-2580-445b-bf35-1bb334f9b304 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:52.821881 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1228/2441] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:38:52 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:38:52.823112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3aa7c6-5d0d-44be-9ec7-4dfa4ab18844 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:52.833387 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3aa7c6-5d0d-44be-9ec7-4dfa4ab18844 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:52.837971 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a574d9b-34a9-4af0-b4b4-f0449d33fc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:52.838621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a574d9b-34a9-4af0-b4b4-f0449d33fc03 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:52.838903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a574d9b-34a9-4af0-b4b4-f0449d33fc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:52.839247 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a574d9b-34a9-4af0-b4b4-f0449d33fc03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:52.841032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a574d9b-34a9-4af0-b4b4-f0449d33fc03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:52.842373 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a574d9b-34a9-4af0-b4b4-f0449d33fc03 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:52.853433 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1214/2442] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:38:52.857489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d1b7d-2580-445b-bf35-1bb334f9b304 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:52.869085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d1b7d-2580-445b-bf35-1bb334f9b304 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:52.874137 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098162e-078f-4b6d-b727-e22e7dae0e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:52.874873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098162e-078f-4b6d-b727-e22e7dae0e50 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:52.875218 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098162e-078f-4b6d-b727-e22e7dae0e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:52.875520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098162e-078f-4b6d-b727-e22e7dae0e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:52.876262 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098162e-078f-4b6d-b727-e22e7dae0e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:52.878007 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098162e-078f-4b6d-b727-e22e7dae0e50 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:52.891117 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1229/2443] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:38:52.895663 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a574d9b-34a9-4af0-b4b4-f0449d33fc03 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:52.905754 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a574d9b-34a9-4af0-b4b4-f0449d33fc03 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:52.910909 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b751bb0-07f4-4140-925d-87eba0332b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:52.911815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b751bb0-07f4-4140-925d-87eba0332b27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:52.912089 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b751bb0-07f4-4140-925d-87eba0332b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:52.912360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b751bb0-07f4-4140-925d-87eba0332b27 admin admin] PATH_INFO: `/v3/projects/2df2b1715762427cb824e381a83bdad1/users/75744cbe26cb4635af3db596f3c235b5/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:52.927223 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b751bb0-07f4-4140-925d-87eba0332b27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2df2b1715762427cb824e381a83bdad1, user_id=75744cbe26cb4635af3db596f3c235b5, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:52.929850 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b751bb0-07f4-4140-925d-87eba0332b27 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:52.942906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1215/2444] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:38:52 2022] PUT /identity/v3/projects/2df2b1715762427cb824e381a83bdad1/users/75744cbe26cb4635af3db596f3c235b5/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:38:52.948216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eec06-f84b-4b83-aadc-ef7f36104022 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:52.948745 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eec06-f84b-4b83-aadc-ef7f36104022 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:52.949247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eec06-f84b-4b83-aadc-ef7f36104022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:52.967467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-268eec06-f84b-4b83-aadc-ef7f36104022 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:38:52.968700 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-268eec06-f84b-4b83-aadc-ef7f36104022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:53.029046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1230/2445] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:38:53.039243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b751bb0-07f4-4140-925d-87eba0332b27 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:53.040967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0b751bb0-07f4-4140-925d-87eba0332b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:53.062319 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:38:53.098798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b751bb0-07f4-4140-925d-87eba0332b27 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:53.102442 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135adbce-49d1-48fd-9e64-9a5013f19153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:53.103101 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135adbce-49d1-48fd-9e64-9a5013f19153 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:53.103382 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135adbce-49d1-48fd-9e64-9a5013f19153 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:53.103656 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135adbce-49d1-48fd-9e64-9a5013f19153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:53.112725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135adbce-49d1-48fd-9e64-9a5013f19153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:53.114149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135adbce-49d1-48fd-9e64-9a5013f19153 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:53.123478 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1216/2446] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:38:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:38:57.107283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4dc0d-49d3-4be8-aa4d-c71bbfbe9c0d None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:57.107283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4dc0d-49d3-4be8-aa4d-c71bbfbe9c0d None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:57.107283 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4dc0d-49d3-4be8-aa4d-c71bbfbe9c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:57.137055 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-ddb4dc0d-49d3-4be8-aa4d-c71bbfbe9c0d None None] MFA Rules not processed for user `75744cbe26cb4635af3db596f3c235b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:38:57.138293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ddb4dc0d-49d3-4be8-aa4d-c71bbfbe9c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:38:57.157447 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:38:57.217294 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1231/2447] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:38:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:38:57.223217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135adbce-49d1-48fd-9e64-9a5013f19153 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:57.236302 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135adbce-49d1-48fd-9e64-9a5013f19153 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:57.241529 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ee25c-b646-47fa-b391-b3c760d481f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:57.242328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ee25c-b646-47fa-b391-b3c760d481f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:57.243036 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ee25c-b646-47fa-b391-b3c760d481f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:57.243397 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ee25c-b646-47fa-b391-b3c760d481f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:57.244180 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ee25c-b646-47fa-b391-b3c760d481f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:57.246430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ee25c-b646-47fa-b391-b3c760d481f5 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:57.282976 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1217/2448] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:38:57 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:38:57.286704 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4dc0d-49d3-4be8-aa4d-c71bbfbe9c0d None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:57.298902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4dc0d-49d3-4be8-aa4d-c71bbfbe9c0d None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:57.358648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b1880-ab1a-41b6-84b0-f8a9b447a550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:57.359490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b1880-ab1a-41b6-84b0-f8a9b447a550 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:57.359837 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b1880-ab1a-41b6-84b0-f8a9b447a550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:57.360861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b1880-ab1a-41b6-84b0-f8a9b447a550 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:57.361633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b1880-ab1a-41b6-84b0-f8a9b447a550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:57.363688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b1880-ab1a-41b6-84b0-f8a9b447a550 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:57.388742 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1232/2449] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:38:57 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:38:57.389938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ee25c-b646-47fa-b391-b3c760d481f5 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:57.401052 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ee25c-b646-47fa-b391-b3c760d481f5 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:57.405973 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ba760-a320-4419-accb-e38c5e08f3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:57.406712 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ba760-a320-4419-accb-e38c5e08f3d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:57.407677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ba760-a320-4419-accb-e38c5e08f3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:57.408016 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ba760-a320-4419-accb-e38c5e08f3d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:57.408818 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ba760-a320-4419-accb-e38c5e08f3d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:57.410515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ba760-a320-4419-accb-e38c5e08f3d4 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:57.423373 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1218/2450] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:38:57.428014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b1880-ab1a-41b6-84b0-f8a9b447a550 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:57.438758 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b1880-ab1a-41b6-84b0-f8a9b447a550 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:57.443965 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba5d2d-3838-426a-9ebc-62ae17cb41dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:57.445014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba5d2d-3838-426a-9ebc-62ae17cb41dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:57.445364 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba5d2d-3838-426a-9ebc-62ae17cb41dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:57.445600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba5d2d-3838-426a-9ebc-62ae17cb41dc admin admin] PATH_INFO: `/v3/projects/3a764b26086c41fc8a4d790eba1ae62a/users/14fc83ff7a094a0da889617d7ab2b178/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:57.462362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba5d2d-3838-426a-9ebc-62ae17cb41dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a764b26086c41fc8a4d790eba1ae62a, user_id=14fc83ff7a094a0da889617d7ab2b178, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:57.465172 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba5d2d-3838-426a-9ebc-62ae17cb41dc admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:57.479860 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1233/2451] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:38:57 2022] PUT /identity/v3/projects/3a764b26086c41fc8a4d790eba1ae62a/users/14fc83ff7a094a0da889617d7ab2b178/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:38:57.483891 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ba760-a320-4419-accb-e38c5e08f3d4 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:57.497821 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ba760-a320-4419-accb-e38c5e08f3d4 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:57.564775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc74d9f-79fd-42b9-90af-11ba9bada0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:57.565453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc74d9f-79fd-42b9-90af-11ba9bada0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:57.565731 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc74d9f-79fd-42b9-90af-11ba9bada0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:57.566006 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc74d9f-79fd-42b9-90af-11ba9bada0d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:57.566746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc74d9f-79fd-42b9-90af-11ba9bada0d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:57.568398 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc74d9f-79fd-42b9-90af-11ba9bada0d9 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:57.579682 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1219/2452] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:38:57.583972 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba5d2d-3838-426a-9ebc-62ae17cb41dc admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:38:57.595224 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba5d2d-3838-426a-9ebc-62ae17cb41dc admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:38:57.599943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c97a88-75c5-4558-a8cc-2c5c8c9c18a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:38:57.600828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c97a88-75c5-4558-a8cc-2c5c8c9c18a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:38:57.601109 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c97a88-75c5-4558-a8cc-2c5c8c9c18a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:38:57.601379 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c97a88-75c5-4558-a8cc-2c5c8c9c18a2 admin admin] PATH_INFO: `/v3/projects/3a764b26086c41fc8a4d790eba1ae62a/users/14fc83ff7a094a0da889617d7ab2b178/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:38:57.603369 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c97a88-75c5-4558-a8cc-2c5c8c9c18a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a764b26086c41fc8a4d790eba1ae62a, user_id=14fc83ff7a094a0da889617d7ab2b178, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:38:57.606690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c97a88-75c5-4558-a8cc-2c5c8c9c18a2 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:38:57.621505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1234/2453] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:38:57 2022] PUT /identity/v3/projects/3a764b26086c41fc8a4d790eba1ae62a/users/14fc83ff7a094a0da889617d7ab2b178/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:39:04.003427 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49425136-e407-433c-a04e-0379b04ec441 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:04.003922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49425136-e407-433c-a04e-0379b04ec441 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:04.004043 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49425136-e407-433c-a04e-0379b04ec441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:04.042069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-49425136-e407-433c-a04e-0379b04ec441 None None] MFA Rules not processed for user `14fc83ff7a094a0da889617d7ab2b178`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:39:04.043854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-49425136-e407-433c-a04e-0379b04ec441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:04.132725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1220/2454] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:39:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:39:04.147291 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c97a88-75c5-4558-a8cc-2c5c8c9c18a2 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:39:04.149155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-88c97a88-75c5-4558-a8cc-2c5c8c9c18a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:04.217580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c97a88-75c5-4558-a8cc-2c5c8c9c18a2 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:39:04.223368 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a0e0c-1aa2-438b-8d78-fcd20c513c00 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b7a38b4a7754037a00d11dafc88645b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:39:04.224138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a0e0c-1aa2-438b-8d78-fcd20c513c00 service glance] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:04.224489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a0e0c-1aa2-438b-8d78-fcd20c513c00 service glance] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:04.224840 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a0e0c-1aa2-438b-8d78-fcd20c513c00 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:04.251486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:39:04.304705 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a0e0c-1aa2-438b-8d78-fcd20c513c00 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:39:04.306893 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a0e0c-1aa2-438b-8d78-fcd20c513c00 service glance] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:39:04.318773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1235/2455] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:39:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:39:04.452611 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d13db-e61e-449e-81b9-faa000d57326 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:04.452926 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d13db-e61e-449e-81b9-faa000d57326 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:04.453466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d13db-e61e-449e-81b9-faa000d57326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:04.469622 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f34357-c005-486a-a236-7c1599fda2a1 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:04.469916 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f34357-c005-486a-a236-7c1599fda2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:04.470200 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f34357-c005-486a-a236-7c1599fda2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:04.476849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-644d13db-e61e-449e-81b9-faa000d57326 None None] MFA Rules not processed for user `4f75b8b484384554ad59aa20535875e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:39:04.478069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-644d13db-e61e-449e-81b9-faa000d57326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:04.484253 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:39:04.490145 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-17f34357-c005-486a-a236-7c1599fda2a1 None None] MFA Rules not processed for user `5cf43634335a428093ee30c2dd99187b`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:39:04.491644 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-17f34357-c005-486a-a236-7c1599fda2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:04.553467 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1221/2456] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 14 20:39:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:39:04.568526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1236/2457] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:39:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:39:04.578150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d13db-e61e-449e-81b9-faa000d57326 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:39:04.579749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-644d13db-e61e-449e-81b9-faa000d57326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:04.656644 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d13db-e61e-449e-81b9-faa000d57326 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:39:04.661948 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc636d1c-9c34-47de-a460-79212702e207 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc23233929144a5b7b3a1733da19828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:39:04.662705 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc636d1c-9c34-47de-a460-79212702e207 service swift] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:04.663071 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc636d1c-9c34-47de-a460-79212702e207 service swift] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:04.663353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc636d1c-9c34-47de-a460-79212702e207 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:04.672103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc636d1c-9c34-47de-a460-79212702e207 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:39:04.674258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc636d1c-9c34-47de-a460-79212702e207 service swift] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:39:04.684646 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1222/2458] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 14 20:39:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 20:39:14.559274 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=41 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:39:14.679961 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=52 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:39:16.516876 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b632c9-889a-46b2-b4c9-9bc1d5218bfd None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:16.517429 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b632c9-889a-46b2-b4c9-9bc1d5218bfd None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:16.517525 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b632c9-889a-46b2-b4c9-9bc1d5218bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:16.538050 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-b6b632c9-889a-46b2-b4c9-9bc1d5218bfd None None] MFA Rules not processed for user `75744cbe26cb4635af3db596f3c235b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:39:16.539407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b6b632c9-889a-46b2-b4c9-9bc1d5218bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:16.547427 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1237/2459] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:39:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:39:18.904947 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e93208-8911-4ac1-afd0-d72acc755619 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:18.906001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e93208-8911-4ac1-afd0-d72acc755619 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:18.906967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e93208-8911-4ac1-afd0-d72acc755619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:18.941913 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e9e93208-8911-4ac1-afd0-d72acc755619 None None] MFA Rules not processed for user `4f75b8b484384554ad59aa20535875e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:39:18.943481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e9e93208-8911-4ac1-afd0-d72acc755619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:18.953793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1223/2460] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 14 20:39:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:39:18.970257 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b632c9-889a-46b2-b4c9-9bc1d5218bfd None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:39:18.993862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b632c9-889a-46b2-b4c9-9bc1d5218bfd None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:39:19.018587 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43919e5e-66fd-4bc0-9b2b-c6968f46e6cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc23233929144a5b7b3a1733da19828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:39:19.019300 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43919e5e-66fd-4bc0-9b2b-c6968f46e6cf service swift] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:19.019581 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43919e5e-66fd-4bc0-9b2b-c6968f46e6cf service swift] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:19.019859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43919e5e-66fd-4bc0-9b2b-c6968f46e6cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:19.039490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43919e5e-66fd-4bc0-9b2b-c6968f46e6cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:39:19.041064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43919e5e-66fd-4bc0-9b2b-c6968f46e6cf service swift] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:39:19.066748 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1238/2461] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 14 20:39:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 20:39:28.925936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:39:29.062278 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=9 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:39:37.242628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e93208-8911-4ac1-afd0-d72acc755619 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:39:37.312251 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e93208-8911-4ac1-afd0-d72acc755619 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:39:37.328318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c18af0-90cc-48e3-a169-8a58acae4fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:39:37.329715 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c18af0-90cc-48e3-a169-8a58acae4fdf service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:37.330270 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c18af0-90cc-48e3-a169-8a58acae4fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:37.330847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c18af0-90cc-48e3-a169-8a58acae4fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:37.333092 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b1c18af0-90cc-48e3-a169-8a58acae4fdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:37.417479 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c18af0-90cc-48e3-a169-8a58acae4fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:39:37.418910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c18af0-90cc-48e3-a169-8a58acae4fdf service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:39:37.431985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1224/2462] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:39:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:39:39.308430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43919e5e-66fd-4bc0-9b2b-c6968f46e6cf service swift] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:39:39.309937 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-43919e5e-66fd-4bc0-9b2b-c6968f46e6cf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:39.381650 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43919e5e-66fd-4bc0-9b2b-c6968f46e6cf service swift] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:39:39.385714 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8f1cd-639a-4a2a-8727-ab34d7340269 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:39:39.386749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8f1cd-639a-4a2a-8727-ab34d7340269 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:39.387239 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8f1cd-639a-4a2a-8727-ab34d7340269 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:39.387699 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8f1cd-639a-4a2a-8727-ab34d7340269 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:39.390466 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-39f8f1cd-639a-4a2a-8727-ab34d7340269 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:39.407566 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8f1cd-639a-4a2a-8727-ab34d7340269 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:39:39.409997 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8f1cd-639a-4a2a-8727-ab34d7340269 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:39:39.425480 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1239/2463] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:39:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:39:43.261177 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd151c1f-dab1-4cc9-a685-16549bf08cd0 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:43.261658 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd151c1f-dab1-4cc9-a685-16549bf08cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:43.261754 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd151c1f-dab1-4cc9-a685-16549bf08cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:43.281089 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-fd151c1f-dab1-4cc9-a685-16549bf08cd0 None None] MFA Rules not processed for user `16c6975c73954487a1f15649693b1d8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:39:43.282326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fd151c1f-dab1-4cc9-a685-16549bf08cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:43.337922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1225/2464] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:39:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:39:44.619828 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8f1cd-639a-4a2a-8727-ab34d7340269 service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:39:44.633799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8f1cd-639a-4a2a-8727-ab34d7340269 service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:39:44.639120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d8914-d850-4eb8-8672-c7bac1a8974b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:39:44.640053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d8914-d850-4eb8-8672-c7bac1a8974b service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:44.640324 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d8914-d850-4eb8-8672-c7bac1a8974b service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:44.640663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d8914-d850-4eb8-8672-c7bac1a8974b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:44.642692 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ec8d8914-d850-4eb8-8672-c7bac1a8974b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:44.705707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d8914-d850-4eb8-8672-c7bac1a8974b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:39:44.707178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d8914-d850-4eb8-8672-c7bac1a8974b service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:39:44.716623 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1240/2465] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:39:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:39:48.555049 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd151c1f-dab1-4cc9-a685-16549bf08cd0 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:39:48.556494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fd151c1f-dab1-4cc9-a685-16549bf08cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:48.593940 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd151c1f-dab1-4cc9-a685-16549bf08cd0 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:39:48.608709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67415777-8feb-4e51-93c5-5ca58bc690ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:39:48.609788 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67415777-8feb-4e51-93c5-5ca58bc690ea service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:48.610306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67415777-8feb-4e51-93c5-5ca58bc690ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:48.610800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67415777-8feb-4e51-93c5-5ca58bc690ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:48.625384 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67415777-8feb-4e51-93c5-5ca58bc690ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:39:48.628280 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67415777-8feb-4e51-93c5-5ca58bc690ea service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:39:48.661775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1226/2466] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:39:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:39:52.383673 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d8914-d850-4eb8-8672-c7bac1a8974b service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:39:52.386046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ec8d8914-d850-4eb8-8672-c7bac1a8974b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:52.429648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d8914-d850-4eb8-8672-c7bac1a8974b service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:39:52.445450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b939758-db97-4d9e-b60e-3d8beb53a066 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:39:52.446673 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b939758-db97-4d9e-b60e-3d8beb53a066 service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:52.447163 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b939758-db97-4d9e-b60e-3d8beb53a066 service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:52.448879 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b939758-db97-4d9e-b60e-3d8beb53a066 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:52.454383 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6b939758-db97-4d9e-b60e-3d8beb53a066 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:52.469564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b939758-db97-4d9e-b60e-3d8beb53a066 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:39:52.471656 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b939758-db97-4d9e-b60e-3d8beb53a066 service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:39:52.490098 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1241/2467] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:39:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:39:53.340568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d478b-3736-4c13-9bf5-4155865c9aea None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:53.340969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d478b-3736-4c13-9bf5-4155865c9aea None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:53.341131 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d478b-3736-4c13-9bf5-4155865c9aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:53.358732 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-9d5d478b-3736-4c13-9bf5-4155865c9aea None None] MFA Rules not processed for user `6502e3ffa3b84333a00ea5f712767d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:39:53.359987 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9d5d478b-3736-4c13-9bf5-4155865c9aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:53.398808 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:39:53.428428 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1227/2468] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:39:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:39:53.802568 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b939758-db97-4d9e-b60e-3d8beb53a066 service nova] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:39:53.812458 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b939758-db97-4d9e-b60e-3d8beb53a066 service nova] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:39:53.816090 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb92d9-858d-4851-b571-39d153836e42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:39:53.816756 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb92d9-858d-4851-b571-39d153836e42 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:53.817040 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb92d9-858d-4851-b571-39d153836e42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:53.817316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb92d9-858d-4851-b571-39d153836e42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:53.819133 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-56bb92d9-858d-4851-b571-39d153836e42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:53.828691 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb92d9-858d-4851-b571-39d153836e42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:39:53.830094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb92d9-858d-4851-b571-39d153836e42 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:39:53.838608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1242/2469] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:39:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:39:54.344098 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d478b-3736-4c13-9bf5-4155865c9aea None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:39:54.345869 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9d5d478b-3736-4c13-9bf5-4155865c9aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:54.433321 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d478b-3736-4c13-9bf5-4155865c9aea None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:39:54.439609 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dce9c8-5fd7-4c19-b1fe-eb0754724673 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afad68819504c2b864851c6989ae25a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:39:54.440349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dce9c8-5fd7-4c19-b1fe-eb0754724673 service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:54.440739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dce9c8-5fd7-4c19-b1fe-eb0754724673 service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:54.441133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dce9c8-5fd7-4c19-b1fe-eb0754724673 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:54.443425 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f1dce9c8-5fd7-4c19-b1fe-eb0754724673 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:54.457371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dce9c8-5fd7-4c19-b1fe-eb0754724673 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:39:54.459609 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dce9c8-5fd7-4c19-b1fe-eb0754724673 service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:39:54.470734 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1228/2470] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:39:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:39:55.593872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb92d9-858d-4851-b571-39d153836e42 service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:39:55.602609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb92d9-858d-4851-b571-39d153836e42 service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:39:55.606361 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf57836-5ab9-488d-be34-cd16c566899f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:39:55.607814 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf57836-5ab9-488d-be34-cd16c566899f service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:55.608137 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf57836-5ab9-488d-be34-cd16c566899f service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:55.608414 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf57836-5ab9-488d-be34-cd16c566899f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:55.610515 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4cf57836-5ab9-488d-be34-cd16c566899f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:39:55.681724 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf57836-5ab9-488d-be34-cd16c566899f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:39:55.683603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf57836-5ab9-488d-be34-cd16c566899f service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:39:55.695556 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1243/2471] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:39:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:39:59.980316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0f5ba-06dc-453a-afd3-500172782980 None None] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:59.980906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0f5ba-06dc-453a-afd3-500172782980 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:59.981503 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0f5ba-06dc-453a-afd3-500172782980 None None] PATH_INFO: `/` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:39:59.984229 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1229/2472] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 14 20:39:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 20:39:59.992240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d986b12-fb83-492d-8928-9a9ed82929d3 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:39:59.992752 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d986b12-fb83-492d-8928-9a9ed82929d3 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:39:59.992867 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d986b12-fb83-492d-8928-9a9ed82929d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:00.011631 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-5d986b12-fb83-492d-8928-9a9ed82929d3 None None] MFA Rules not processed for user `1afad68819504c2b864851c6989ae25a`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:40:00.023122 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-5d986b12-fb83-492d-8928-9a9ed82929d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:00.044158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1244/2473] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 14 20:39:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:40:00.057442 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0f5ba-06dc-453a-afd3-500172782980 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:00.059361 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3bd0f5ba-06dc-453a-afd3-500172782980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:00.068115 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d986b12-fb83-492d-8928-9a9ed82929d3 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:00.069604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-5d986b12-fb83-492d-8928-9a9ed82929d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:00.077793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:40:00.086782 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d986b12-fb83-492d-8928-9a9ed82929d3 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:00.087930 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0f5ba-06dc-453a-afd3-500172782980 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:00.093810 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c362f5-cb40-4685-8bbb-dd12f03f23e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afad68819504c2b864851c6989ae25a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:00.094918 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e588fe9-2525-4462-a0dc-c4f6ec1e9862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:00.095216 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c362f5-cb40-4685-8bbb-dd12f03f23e2 service placement] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:00.095741 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c362f5-cb40-4685-8bbb-dd12f03f23e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:00.096073 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c362f5-cb40-4685-8bbb-dd12f03f23e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:00.096386 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e588fe9-2525-4462-a0dc-c4f6ec1e9862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:00.096668 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e588fe9-2525-4462-a0dc-c4f6ec1e9862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:00.096941 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e588fe9-2525-4462-a0dc-c4f6ec1e9862 admin admin] PATH_INFO: `/v3/users/765c911957bd4ca68dd520fc4fbba1c9` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:00.098233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e588fe9-2525-4462-a0dc-c4f6ec1e9862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=765c911957bd4ca68dd520fc4fbba1c9)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:00.100485 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e588fe9-2525-4462-a0dc-c4f6ec1e9862 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:00.108416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c362f5-cb40-4685-8bbb-dd12f03f23e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:00.110067 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c362f5-cb40-4685-8bbb-dd12f03f23e2 service placement] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:00.122069 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1245/2474] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:40:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:40:00.158592 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9e588fe9-2525-4462-a0dc-c4f6ec1e9862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '765c911957bd4ca68dd520fc4fbba1c9'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:40:00.166881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9e588fe9-2525-4462-a0dc-c4f6ec1e9862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '765c911957bd4ca68dd520fc4fbba1c9'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:40:00.167637 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9e588fe9-2525-4462-a0dc-c4f6ec1e9862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '765c911957bd4ca68dd520fc4fbba1c9'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:40:00.168289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9e588fe9-2525-4462-a0dc-c4f6ec1e9862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '765c911957bd4ca68dd520fc4fbba1c9'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:40:00.181070 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9e588fe9-2525-4462-a0dc-c4f6ec1e9862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '765c911957bd4ca68dd520fc4fbba1c9'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:40:00.193102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1230/2475] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:40:00 2022] DELETE /identity/v3/users/765c911957bd4ca68dd520fc4fbba1c9 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:40:00.365540 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c362f5-cb40-4685-8bbb-dd12f03f23e2 service placement] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:00.379002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c362f5-cb40-4685-8bbb-dd12f03f23e2 service placement] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:00.445813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc69f4b-d87b-4763-873c-1ba706b920a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:00.446689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc69f4b-d87b-4763-873c-1ba706b920a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:00.447095 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc69f4b-d87b-4763-873c-1ba706b920a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:00.447275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc69f4b-d87b-4763-873c-1ba706b920a4 admin admin] PATH_INFO: `/v3/projects/cceb4db55bba4bd099104acdcb6d2762` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:00.448548 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc69f4b-d87b-4763-873c-1ba706b920a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cceb4db55bba4bd099104acdcb6d2762)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:00.450412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc69f4b-d87b-4763-873c-1ba706b920a4 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:00.499877 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ccc69f4b-d87b-4763-873c-1ba706b920a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cceb4db55bba4bd099104acdcb6d2762', 'request_id': 'req-ccc69f4b-d87b-4763-873c-1ba706b920a4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:40:00.508686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ccc69f4b-d87b-4763-873c-1ba706b920a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cceb4db55bba4bd099104acdcb6d2762', 'request_id': 'req-ccc69f4b-d87b-4763-873c-1ba706b920a4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:40:00.515799 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ccc69f4b-d87b-4763-873c-1ba706b920a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cceb4db55bba4bd099104acdcb6d2762', 'request_id': 'req-ccc69f4b-d87b-4763-873c-1ba706b920a4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:40:00.527166 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ccc69f4b-d87b-4763-873c-1ba706b920a4 admin admin] The token cache is being invalidate because project cceb4db55bba4bd099104acdcb6d2762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:40:00.527590 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ccc69f4b-d87b-4763-873c-1ba706b920a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:40:00.528320 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ccc69f4b-d87b-4763-873c-1ba706b920a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:40:00.530664 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1246/2476] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:40:00 2022] DELETE /identity/v3/projects/cceb4db55bba4bd099104acdcb6d2762 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:40:00.536733 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be30749-270e-42f5-8217-6d2946250036 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:00.537078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be30749-270e-42f5-8217-6d2946250036 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:00.537363 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be30749-270e-42f5-8217-6d2946250036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:00.555368 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-0be30749-270e-42f5-8217-6d2946250036 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:40:00.556599 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0be30749-270e-42f5-8217-6d2946250036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:00.593366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:40:00.626760 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1231/2477] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:40:00.634275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521fb0a-9f1e-4e66-b3ce-ce7d639a1c13 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:00.634564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521fb0a-9f1e-4e66-b3ce-ce7d639a1c13 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:00.634841 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521fb0a-9f1e-4e66-b3ce-ce7d639a1c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:00.656453 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-f521fb0a-9f1e-4e66-b3ce-ce7d639a1c13 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:40:00.657314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f521fb0a-9f1e-4e66-b3ce-ce7d639a1c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:00.665780 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1247/2478] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:40:00.669688 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be30749-270e-42f5-8217-6d2946250036 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:00.680779 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be30749-270e-42f5-8217-6d2946250036 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:00.686734 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e0429-4657-4d79-a11d-6d51fbe4f88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:00.687528 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e0429-4657-4d79-a11d-6d51fbe4f88e admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:00.687898 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e0429-4657-4d79-a11d-6d51fbe4f88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:00.688245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e0429-4657-4d79-a11d-6d51fbe4f88e admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:00.689072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e0429-4657-4d79-a11d-6d51fbe4f88e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:40:00.689384 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e0429-4657-4d79-a11d-6d51fbe4f88e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:00.691024 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e0429-4657-4d79-a11d-6d51fbe4f88e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:00.708302 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1232/2479] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:40:00.709507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521fb0a-9f1e-4e66-b3ce-ce7d639a1c13 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:00.719748 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521fb0a-9f1e-4e66-b3ce-ce7d639a1c13 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:00.724307 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eebd33-928e-41e5-97b6-0e26b17f0da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:00.725033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eebd33-928e-41e5-97b6-0e26b17f0da4 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:00.725381 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eebd33-928e-41e5-97b6-0e26b17f0da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:00.725721 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eebd33-928e-41e5-97b6-0e26b17f0da4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:00.726489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eebd33-928e-41e5-97b6-0e26b17f0da4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:00.728481 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eebd33-928e-41e5-97b6-0e26b17f0da4 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:00.753415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1248/2480] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:40:00 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:40:00.757471 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e0429-4657-4d79-a11d-6d51fbe4f88e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:00.768484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e0429-4657-4d79-a11d-6d51fbe4f88e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:00.834972 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef7f61-9f2c-430c-bcc8-8444df3db888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:00.835674 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef7f61-9f2c-430c-bcc8-8444df3db888 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:00.836097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef7f61-9f2c-430c-bcc8-8444df3db888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:00.836381 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef7f61-9f2c-430c-bcc8-8444df3db888 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:00.837078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef7f61-9f2c-430c-bcc8-8444df3db888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:00.839036 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef7f61-9f2c-430c-bcc8-8444df3db888 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:00.861028 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1233/2481] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:40:00 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:40:00.865465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eebd33-928e-41e5-97b6-0e26b17f0da4 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:00.877070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eebd33-928e-41e5-97b6-0e26b17f0da4 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:00.881609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8fd92-5b9f-4377-8a65-f31752c0e522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:00.882286 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8fd92-5b9f-4377-8a65-f31752c0e522 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:00.882567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8fd92-5b9f-4377-8a65-f31752c0e522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:00.882844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8fd92-5b9f-4377-8a65-f31752c0e522 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:00.883689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8fd92-5b9f-4377-8a65-f31752c0e522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:00.885308 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8fd92-5b9f-4377-8a65-f31752c0e522 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:00.896851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1249/2482] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:40:00.900913 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef7f61-9f2c-430c-bcc8-8444df3db888 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:00.913343 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef7f61-9f2c-430c-bcc8-8444df3db888 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:00.918522 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f856500-c172-496b-8527-8ef7ffaa5526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:00.919224 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f856500-c172-496b-8527-8ef7ffaa5526 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:00.919510 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f856500-c172-496b-8527-8ef7ffaa5526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:00.919788 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f856500-c172-496b-8527-8ef7ffaa5526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:00.920521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f856500-c172-496b-8527-8ef7ffaa5526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:00.922159 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f856500-c172-496b-8527-8ef7ffaa5526 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:00.932484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1234/2483] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:40:00.937307 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8fd92-5b9f-4377-8a65-f31752c0e522 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:00.947246 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8fd92-5b9f-4377-8a65-f31752c0e522 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:00.953311 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61338bd1-960e-4f8b-95c5-70baea517b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:00.954182 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61338bd1-960e-4f8b-95c5-70baea517b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:00.954463 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61338bd1-960e-4f8b-95c5-70baea517b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:00.954736 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61338bd1-960e-4f8b-95c5-70baea517b7c admin admin] PATH_INFO: `/v3/projects/01d544a1fa8b4ac08d4588eab1a84ff5/users/813a2d3222f54b18a4322174a409183e/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:00.969571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61338bd1-960e-4f8b-95c5-70baea517b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=01d544a1fa8b4ac08d4588eab1a84ff5, user_id=813a2d3222f54b18a4322174a409183e, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:00.972286 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61338bd1-960e-4f8b-95c5-70baea517b7c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:00.984737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1250/2484] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:40:00 2022] PUT /identity/v3/projects/01d544a1fa8b4ac08d4588eab1a84ff5/users/813a2d3222f54b18a4322174a409183e/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:40:00.989771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e948b6-3b52-4e36-a6c4-7e24db61e27b None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:00.990211 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e948b6-3b52-4e36-a6c4-7e24db61e27b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:00.990362 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e948b6-3b52-4e36-a6c4-7e24db61e27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:01.008295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-19e948b6-3b52-4e36-a6c4-7e24db61e27b None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:40:01.009511 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-19e948b6-3b52-4e36-a6c4-7e24db61e27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:01.070642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1235/2485] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:40:01.081097 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61338bd1-960e-4f8b-95c5-70baea517b7c admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:01.082962 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-61338bd1-960e-4f8b-95c5-70baea517b7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:01.101499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:40:01.143603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61338bd1-960e-4f8b-95c5-70baea517b7c admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:01.147456 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6a118-b4cc-4c4b-ac33-9706ed43ce62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:01.148100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6a118-b4cc-4c4b-ac33-9706ed43ce62 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:01.148383 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6a118-b4cc-4c4b-ac33-9706ed43ce62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:01.148658 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6a118-b4cc-4c4b-ac33-9706ed43ce62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:01.157651 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6a118-b4cc-4c4b-ac33-9706ed43ce62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:01.159102 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6a118-b4cc-4c4b-ac33-9706ed43ce62 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:01.168568 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1251/2486] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:40:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:40:02.720462 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e948b6-3b52-4e36-a6c4-7e24db61e27b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:02.722045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-19e948b6-3b52-4e36-a6c4-7e24db61e27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:02.807141 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:40:02.852049 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e948b6-3b52-4e36-a6c4-7e24db61e27b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:02.859218 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853575f-2102-4789-ac7d-cd3cbc6c06b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:02.860599 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853575f-2102-4789-ac7d-cd3cbc6c06b8 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:02.860897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853575f-2102-4789-ac7d-cd3cbc6c06b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:02.861173 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853575f-2102-4789-ac7d-cd3cbc6c06b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:02.863262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1853575f-2102-4789-ac7d-cd3cbc6c06b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:02.952903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853575f-2102-4789-ac7d-cd3cbc6c06b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:02.955185 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853575f-2102-4789-ac7d-cd3cbc6c06b8 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:02.992596 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1236/2487] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:40:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:40:03.636945 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6a118-b4cc-4c4b-ac33-9706ed43ce62 service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:03.639656 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-78a6a118-b4cc-4c4b-ac33-9706ed43ce62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:03.665232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6a118-b4cc-4c4b-ac33-9706ed43ce62 service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:03.678327 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e039c-7637-4062-9870-1cd8cbf97156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:03.679288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e039c-7637-4062-9870-1cd8cbf97156 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:03.679735 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e039c-7637-4062-9870-1cd8cbf97156 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:03.680172 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e039c-7637-4062-9870-1cd8cbf97156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:03.685699 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6d0e039c-7637-4062-9870-1cd8cbf97156 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:03.702486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e039c-7637-4062-9870-1cd8cbf97156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:03.704220 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e039c-7637-4062-9870-1cd8cbf97156 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:03.715759 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1252/2488] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:40:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:40:06.627461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca0634-7661-40e9-83e9-6542aa768c7f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:06.627930 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca0634-7661-40e9-83e9-6542aa768c7f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:06.628123 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca0634-7661-40e9-83e9-6542aa768c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:06.657076 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-6dca0634-7661-40e9-83e9-6542aa768c7f None None] MFA Rules not processed for user `813a2d3222f54b18a4322174a409183e`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:40:06.658485 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6dca0634-7661-40e9-83e9-6542aa768c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:06.738017 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1237/2489] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:40:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:40:06.753514 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e039c-7637-4062-9870-1cd8cbf97156 service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:06.754985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6d0e039c-7637-4062-9870-1cd8cbf97156 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:06.772366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e039c-7637-4062-9870-1cd8cbf97156 service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:06.784450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b95488-6855-42b3-a484-d4abbb96f62c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:06.785450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b95488-6855-42b3-a484-d4abbb96f62c service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:06.786123 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b95488-6855-42b3-a484-d4abbb96f62c service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:06.786404 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b95488-6855-42b3-a484-d4abbb96f62c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:06.796559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b95488-6855-42b3-a484-d4abbb96f62c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:06.798203 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b95488-6855-42b3-a484-d4abbb96f62c service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:06.808535 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1253/2490] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:40:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:40:08.719757 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109141cd-38a1-4971-9d13-160a9d204b32 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:08.720230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109141cd-38a1-4971-9d13-160a9d204b32 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:08.720313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109141cd-38a1-4971-9d13-160a9d204b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:08.742426 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-109141cd-38a1-4971-9d13-160a9d204b32 None None] MFA Rules not processed for user `813a2d3222f54b18a4322174a409183e`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:40:08.743729 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-109141cd-38a1-4971-9d13-160a9d204b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:08.751765 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1238/2491] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:40:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:40:12.947099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b95488-6855-42b3-a484-d4abbb96f62c service nova] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:12.960627 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b95488-6855-42b3-a484-d4abbb96f62c service nova] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:12.968594 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93192f-466a-4656-9426-c9fd7109a9be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:12.969534 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93192f-466a-4656-9426-c9fd7109a9be service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:12.969982 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93192f-466a-4656-9426-c9fd7109a9be service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:12.970416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93192f-466a-4656-9426-c9fd7109a9be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:12.972687 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-af93192f-466a-4656-9426-c9fd7109a9be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:13.005575 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93192f-466a-4656-9426-c9fd7109a9be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:13.007732 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93192f-466a-4656-9426-c9fd7109a9be service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:13.031942 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1254/2492] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:40:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:40:13.079420 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109141cd-38a1-4971-9d13-160a9d204b32 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:13.100642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109141cd-38a1-4971-9d13-160a9d204b32 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:13.126410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1411179-23b7-4ae9-b8d1-e8fbcde7fb27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:13.127320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1411179-23b7-4ae9-b8d1-e8fbcde7fb27 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:13.127881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1411179-23b7-4ae9-b8d1-e8fbcde7fb27 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:13.128232 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1411179-23b7-4ae9-b8d1-e8fbcde7fb27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:13.130697 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a1411179-23b7-4ae9-b8d1-e8fbcde7fb27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:13.145903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1411179-23b7-4ae9-b8d1-e8fbcde7fb27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:13.148142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1411179-23b7-4ae9-b8d1-e8fbcde7fb27 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:13.163225 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1239/2493] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:40:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:40:23.024379 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=37 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:40:23.156430 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=76 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:40:24.233533 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93192f-466a-4656-9426-c9fd7109a9be service nova] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:24.264856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93192f-466a-4656-9426-c9fd7109a9be service nova] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:24.280973 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ccfcf-f687-44f8-99e9-6aa9777d97eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:24.284796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ccfcf-f687-44f8-99e9-6aa9777d97eb service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:24.285565 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ccfcf-f687-44f8-99e9-6aa9777d97eb service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:24.287219 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ccfcf-f687-44f8-99e9-6aa9777d97eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:24.294359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-bf4ccfcf-f687-44f8-99e9-6aa9777d97eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:24.320611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ccfcf-f687-44f8-99e9-6aa9777d97eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:24.322918 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ccfcf-f687-44f8-99e9-6aa9777d97eb service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:24.337842 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1255/2494] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:40:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:40:24.471245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352ac57-1a56-4ab2-98ab-421d6fcb214d None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:24.471749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352ac57-1a56-4ab2-98ab-421d6fcb214d None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:24.471945 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352ac57-1a56-4ab2-98ab-421d6fcb214d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:24.491736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-3352ac57-1a56-4ab2-98ab-421d6fcb214d None None] MFA Rules not processed for user `4f75b8b484384554ad59aa20535875e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:40:24.493009 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3352ac57-1a56-4ab2-98ab-421d6fcb214d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:24.563363 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1240/2495] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 14 20:40:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:40:24.620094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ccfcf-f687-44f8-99e9-6aa9777d97eb service nova] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:24.621734 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-bf4ccfcf-f687-44f8-99e9-6aa9777d97eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:24.709290 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ccfcf-f687-44f8-99e9-6aa9777d97eb service nova] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:24.714743 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7385b-4e9c-494d-82e2-b39a199267ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc23233929144a5b7b3a1733da19828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:24.715446 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7385b-4e9c-494d-82e2-b39a199267ca service swift] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:24.715733 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7385b-4e9c-494d-82e2-b39a199267ca service swift] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:24.716011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7385b-4e9c-494d-82e2-b39a199267ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:24.726421 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7385b-4e9c-494d-82e2-b39a199267ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:24.728115 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7385b-4e9c-494d-82e2-b39a199267ca service swift] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:24.741056 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1256/2496] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 14 20:40:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 20:40:34.552146 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=25 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:40:34.734361 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=38 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:40:41.130111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e18d00-1384-42fa-bc62-367d20cbe286 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:41.130582 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e18d00-1384-42fa-bc62-367d20cbe286 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:41.130676 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e18d00-1384-42fa-bc62-367d20cbe286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:41.149722 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f4e18d00-1384-42fa-bc62-367d20cbe286 None None] MFA Rules not processed for user `4f75b8b484384554ad59aa20535875e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:40:41.150957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f4e18d00-1384-42fa-bc62-367d20cbe286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:41.158360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1241/2497] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 14 20:40:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:40:41.170027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7385b-4e9c-494d-82e2-b39a199267ca service swift] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:41.180459 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7385b-4e9c-494d-82e2-b39a199267ca service swift] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:41.185959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b027d8f-5df1-4d46-85bf-7f828f258440 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc23233929144a5b7b3a1733da19828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:41.186596 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b027d8f-5df1-4d46-85bf-7f828f258440 service swift] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:41.186875 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b027d8f-5df1-4d46-85bf-7f828f258440 service swift] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:41.187182 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b027d8f-5df1-4d46-85bf-7f828f258440 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:41.195171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b027d8f-5df1-4d46-85bf-7f828f258440 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:41.196738 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b027d8f-5df1-4d46-85bf-7f828f258440 service swift] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:41.205809 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1257/2498] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 14 20:40:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 20:40:51.144175 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:40:51.202306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=6 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:40:58.121667 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466c332-5818-4807-82cb-a856b1aaab7c None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:58.122147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466c332-5818-4807-82cb-a856b1aaab7c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:58.122242 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466c332-5818-4807-82cb-a856b1aaab7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:58.162095 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1466c332-5818-4807-82cb-a856b1aaab7c None None] MFA Rules not processed for user `4f75b8b484384554ad59aa20535875e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:40:58.163343 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1466c332-5818-4807-82cb-a856b1aaab7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:40:58.190708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1242/2499] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jul 14 20:40:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:40:58.203839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b027d8f-5df1-4d46-85bf-7f828f258440 service swift] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:40:58.247749 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b027d8f-5df1-4d46-85bf-7f828f258440 service swift] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:40:58.252774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c63206-9e66-442c-9b1b-29a25d94b4ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc23233929144a5b7b3a1733da19828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:40:58.253420 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c63206-9e66-442c-9b1b-29a25d94b4ef service swift] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:40:58.253702 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c63206-9e66-442c-9b1b-29a25d94b4ef service swift] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:40:58.253995 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c63206-9e66-442c-9b1b-29a25d94b4ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:40:58.263044 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c63206-9e66-442c-9b1b-29a25d94b4ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:40:58.264630 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c63206-9e66-442c-9b1b-29a25d94b4ef service swift] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:40:58.273079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1258/2500] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 14 20:40:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 20:41:08.188728 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=20 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:08.269790 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=20 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:09.928063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466c332-5818-4807-82cb-a856b1aaab7c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:09.947358 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466c332-5818-4807-82cb-a856b1aaab7c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:09.950907 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d3384-28c2-41f6-8597-41ec7102d9cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:09.951564 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d3384-28c2-41f6-8597-41ec7102d9cf service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:09.951843 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d3384-28c2-41f6-8597-41ec7102d9cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:09.952114 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d3384-28c2-41f6-8597-41ec7102d9cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:09.953863 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5b2d3384-28c2-41f6-8597-41ec7102d9cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:09.987640 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d3384-28c2-41f6-8597-41ec7102d9cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:09.989078 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d3384-28c2-41f6-8597-41ec7102d9cf service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:10.008589 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1243/2501] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:41:13.287445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c63206-9e66-442c-9b1b-29a25d94b4ef service swift] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:13.289745 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d9c63206-9e66-442c-9b1b-29a25d94b4ef service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:13.305991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c63206-9e66-442c-9b1b-29a25d94b4ef service swift] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:13.311441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3aa85b-296b-464e-a2b7-fcd97331d2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:13.312332 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3aa85b-296b-464e-a2b7-fcd97331d2a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:13.312611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3aa85b-296b-464e-a2b7-fcd97331d2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:13.312887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3aa85b-296b-464e-a2b7-fcd97331d2a1 admin admin] PATH_INFO: `/v3/users/16c6975c73954487a1f15649693b1d8a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:13.314089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3aa85b-296b-464e-a2b7-fcd97331d2a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16c6975c73954487a1f15649693b1d8a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:13.316081 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3aa85b-296b-464e-a2b7-fcd97331d2a1 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:13.377838 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0b3aa85b-296b-464e-a2b7-fcd97331d2a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16c6975c73954487a1f15649693b1d8a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:13.392656 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0b3aa85b-296b-464e-a2b7-fcd97331d2a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16c6975c73954487a1f15649693b1d8a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:13.401903 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0b3aa85b-296b-464e-a2b7-fcd97331d2a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16c6975c73954487a1f15649693b1d8a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:13.402802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0b3aa85b-296b-464e-a2b7-fcd97331d2a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16c6975c73954487a1f15649693b1d8a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:13.403677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0b3aa85b-296b-464e-a2b7-fcd97331d2a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16c6975c73954487a1f15649693b1d8a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:13.422764 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1259/2502] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:41:13 2022] DELETE /identity/v3/users/16c6975c73954487a1f15649693b1d8a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:13.622957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d3384-28c2-41f6-8597-41ec7102d9cf service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:13.638474 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d3384-28c2-41f6-8597-41ec7102d9cf service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:13.706662 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca5c3e-8b0b-4d72-a921-2089da173019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:13.707581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca5c3e-8b0b-4d72-a921-2089da173019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:13.707863 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca5c3e-8b0b-4d72-a921-2089da173019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:13.708139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca5c3e-8b0b-4d72-a921-2089da173019 admin admin] PATH_INFO: `/v3/projects/40baf66113cf4fb186ddd5e954a9b7a4` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:13.709372 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca5c3e-8b0b-4d72-a921-2089da173019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40baf66113cf4fb186ddd5e954a9b7a4)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:13.711340 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca5c3e-8b0b-4d72-a921-2089da173019 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:13.777432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-abca5c3e-8b0b-4d72-a921-2089da173019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40baf66113cf4fb186ddd5e954a9b7a4', 'request_id': 'req-abca5c3e-8b0b-4d72-a921-2089da173019'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:13.786827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-abca5c3e-8b0b-4d72-a921-2089da173019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40baf66113cf4fb186ddd5e954a9b7a4', 'request_id': 'req-abca5c3e-8b0b-4d72-a921-2089da173019'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:13.796144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-abca5c3e-8b0b-4d72-a921-2089da173019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40baf66113cf4fb186ddd5e954a9b7a4', 'request_id': 'req-abca5c3e-8b0b-4d72-a921-2089da173019'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:13.812759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-abca5c3e-8b0b-4d72-a921-2089da173019 admin admin] The token cache is being invalidate because project 40baf66113cf4fb186ddd5e954a9b7a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:41:13.813064 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-abca5c3e-8b0b-4d72-a921-2089da173019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:13.813740 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-abca5c3e-8b0b-4d72-a921-2089da173019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:13.815372 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1244/2503] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:41:13 2022] DELETE /identity/v3/projects/40baf66113cf4fb186ddd5e954a9b7a4 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:13.821646 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80efe1-cf9d-40e3-a69c-b33b5cc4bb09 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:13.821794 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80efe1-cf9d-40e3-a69c-b33b5cc4bb09 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:13.822046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80efe1-cf9d-40e3-a69c-b33b5cc4bb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:13.899497 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-bc80efe1-cf9d-40e3-a69c-b33b5cc4bb09 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:13.900786 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-bc80efe1-cf9d-40e3-a69c-b33b5cc4bb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:13.966103 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1260/2504] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:13.974302 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fe9e3-3278-41e2-ae1b-183258d9b1fb None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:13.974522 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fe9e3-3278-41e2-ae1b-183258d9b1fb None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:13.974789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fe9e3-3278-41e2-ae1b-183258d9b1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:13.993906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-ba1fe9e3-3278-41e2-ae1b-183258d9b1fb None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:13.995147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ba1fe9e3-3278-41e2-ae1b-183258d9b1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:14.003852 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1245/2505] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:14.008303 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80efe1-cf9d-40e3-a69c-b33b5cc4bb09 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:14.019266 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80efe1-cf9d-40e3-a69c-b33b5cc4bb09 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:14.024439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df7359-dd1d-4f6b-ac61-bd7980c57f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:14.025093 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df7359-dd1d-4f6b-ac61-bd7980c57f50 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:14.025372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df7359-dd1d-4f6b-ac61-bd7980c57f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:14.025672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df7359-dd1d-4f6b-ac61-bd7980c57f50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:14.026414 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df7359-dd1d-4f6b-ac61-bd7980c57f50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:41:14.026717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df7359-dd1d-4f6b-ac61-bd7980c57f50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:14.028910 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df7359-dd1d-4f6b-ac61-bd7980c57f50 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:14.042131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1261/2506] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:41:14.047311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fe9e3-3278-41e2-ae1b-183258d9b1fb None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:14.056928 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fe9e3-3278-41e2-ae1b-183258d9b1fb None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:14.062148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00956e86-c7a5-4809-a4da-7a822282e2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:14.062824 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00956e86-c7a5-4809-a4da-7a822282e2c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:14.063147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00956e86-c7a5-4809-a4da-7a822282e2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:14.063431 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00956e86-c7a5-4809-a4da-7a822282e2c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:14.064145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00956e86-c7a5-4809-a4da-7a822282e2c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:14.066057 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00956e86-c7a5-4809-a4da-7a822282e2c3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:14.104153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1246/2507] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:41:14 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:41:14.110017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df7359-dd1d-4f6b-ac61-bd7980c57f50 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:14.121227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df7359-dd1d-4f6b-ac61-bd7980c57f50 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:14.155961 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:14.195654 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c1a70-6f62-4b0a-bdab-48325cf54ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:14.196374 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c1a70-6f62-4b0a-bdab-48325cf54ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:14.196659 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c1a70-6f62-4b0a-bdab-48325cf54ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:14.196986 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c1a70-6f62-4b0a-bdab-48325cf54ff4 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:14.197750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c1a70-6f62-4b0a-bdab-48325cf54ff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:14.199791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c1a70-6f62-4b0a-bdab-48325cf54ff4 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:14.219903 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1262/2508] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:41:14 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:41:14.224535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00956e86-c7a5-4809-a4da-7a822282e2c3 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:14.230628 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:14.236632 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00956e86-c7a5-4809-a4da-7a822282e2c3 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:14.242626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e79d8-b783-42cf-9487-dc2a8fa69cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:14.243335 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e79d8-b783-42cf-9487-dc2a8fa69cba admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:14.243695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e79d8-b783-42cf-9487-dc2a8fa69cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:14.244157 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e79d8-b783-42cf-9487-dc2a8fa69cba admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:14.244891 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e79d8-b783-42cf-9487-dc2a8fa69cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:14.246503 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e79d8-b783-42cf-9487-dc2a8fa69cba admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:14.258200 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1247/2509] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:14.262382 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c1a70-6f62-4b0a-bdab-48325cf54ff4 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:14.273209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c1a70-6f62-4b0a-bdab-48325cf54ff4 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:14.278552 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129ed31-a63a-4167-ab48-5a22a5e90911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:14.279373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129ed31-a63a-4167-ab48-5a22a5e90911 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:14.279658 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129ed31-a63a-4167-ab48-5a22a5e90911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:14.280079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129ed31-a63a-4167-ab48-5a22a5e90911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:14.280844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6129ed31-a63a-4167-ab48-5a22a5e90911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:14.282473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6129ed31-a63a-4167-ab48-5a22a5e90911 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:14.293871 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1263/2510] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:14.298126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e79d8-b783-42cf-9487-dc2a8fa69cba admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:14.308880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e79d8-b783-42cf-9487-dc2a8fa69cba admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:14.315390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d252bea-7e27-459d-b391-e72b1cba7f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:14.315390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d252bea-7e27-459d-b391-e72b1cba7f86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:14.315390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d252bea-7e27-459d-b391-e72b1cba7f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:14.316366 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d252bea-7e27-459d-b391-e72b1cba7f86 admin admin] PATH_INFO: `/v3/projects/e6c03c3aff534d7d8b73d3199d925c09/users/1a43aecd3f964e51add35244d3d88a19/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:14.331875 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d252bea-7e27-459d-b391-e72b1cba7f86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6c03c3aff534d7d8b73d3199d925c09, user_id=1a43aecd3f964e51add35244d3d88a19, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:14.334568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d252bea-7e27-459d-b391-e72b1cba7f86 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:14.348389 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1248/2511] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:41:14 2022] PUT /identity/v3/projects/e6c03c3aff534d7d8b73d3199d925c09/users/1a43aecd3f964e51add35244d3d88a19/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:14.360274 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a55319-f0b0-4862-946f-b04d3e4a0f97 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:14.360565 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a55319-f0b0-4862-946f-b04d3e4a0f97 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:14.360834 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a55319-f0b0-4862-946f-b04d3e4a0f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:14.387958 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-a4a55319-f0b0-4862-946f-b04d3e4a0f97 None None] MFA Rules not processed for user `1a43aecd3f964e51add35244d3d88a19`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:14.389170 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-a4a55319-f0b0-4862-946f-b04d3e4a0f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:14.460429 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1264/2512] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:14.472914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d252bea-7e27-459d-b391-e72b1cba7f86 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:14.474649 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-0d252bea-7e27-459d-b391-e72b1cba7f86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:14.557578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d252bea-7e27-459d-b391-e72b1cba7f86 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:14.565015 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da20d5-03a3-4157-9661-22b8aec21a2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:14.565708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da20d5-03a3-4157-9661-22b8aec21a2c service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:14.565960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da20d5-03a3-4157-9661-22b8aec21a2c service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:14.566234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da20d5-03a3-4157-9661-22b8aec21a2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:14.574719 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da20d5-03a3-4157-9661-22b8aec21a2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:14.576351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da20d5-03a3-4157-9661-22b8aec21a2c service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:14.585413 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1249/2513] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:41:17.287187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a55319-f0b0-4862-946f-b04d3e4a0f97 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:17.346487 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a55319-f0b0-4862-946f-b04d3e4a0f97 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:17.351056 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ba9ca-8c00-48fd-b3c3-d0727ae94485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:17.351926 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ba9ca-8c00-48fd-b3c3-d0727ae94485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:17.352208 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ba9ca-8c00-48fd-b3c3-d0727ae94485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:17.352482 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ba9ca-8c00-48fd-b3c3-d0727ae94485 admin admin] PATH_INFO: `/v3/users/1a43aecd3f964e51add35244d3d88a19` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:17.353675 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ba9ca-8c00-48fd-b3c3-d0727ae94485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a43aecd3f964e51add35244d3d88a19)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:17.356229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ba9ca-8c00-48fd-b3c3-d0727ae94485 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:17.395741 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:17.403492 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d98ba9ca-8c00-48fd-b3c3-d0727ae94485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a43aecd3f964e51add35244d3d88a19'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:17.411094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d98ba9ca-8c00-48fd-b3c3-d0727ae94485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a43aecd3f964e51add35244d3d88a19'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:17.417863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d98ba9ca-8c00-48fd-b3c3-d0727ae94485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a43aecd3f964e51add35244d3d88a19'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:17.418513 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d98ba9ca-8c00-48fd-b3c3-d0727ae94485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a43aecd3f964e51add35244d3d88a19'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:17.419146 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d98ba9ca-8c00-48fd-b3c3-d0727ae94485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a43aecd3f964e51add35244d3d88a19'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:17.437251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1265/2514] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:41:17 2022] DELETE /identity/v3/users/1a43aecd3f964e51add35244d3d88a19 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:17.441625 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36ba44-b6c7-4a80-9982-e4c906a83e19 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:17.442032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36ba44-b6c7-4a80-9982-e4c906a83e19 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:17.442177 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36ba44-b6c7-4a80-9982-e4c906a83e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:17.460681 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1b36ba44-b6c7-4a80-9982-e4c906a83e19 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:17.461899 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1b36ba44-b6c7-4a80-9982-e4c906a83e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:17.530800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1250/2515] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:17.540763 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ba9ca-8c00-48fd-b3c3-d0727ae94485 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:17.542203 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d98ba9ca-8c00-48fd-b3c3-d0727ae94485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:17.615071 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ba9ca-8c00-48fd-b3c3-d0727ae94485 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:17.618721 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d0996-933d-4b3d-ba19-d9db611a4a90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:17.619471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d0996-933d-4b3d-ba19-d9db611a4a90 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:17.619756 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d0996-933d-4b3d-ba19-d9db611a4a90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:17.620031 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d0996-933d-4b3d-ba19-d9db611a4a90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:17.629555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d0996-933d-4b3d-ba19-d9db611a4a90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:17.631025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d0996-933d-4b3d-ba19-d9db611a4a90 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:17.642350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1266/2516] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:41:17.931278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36ba44-b6c7-4a80-9982-e4c906a83e19 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:17.932822 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1b36ba44-b6c7-4a80-9982-e4c906a83e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:17.945878 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36ba44-b6c7-4a80-9982-e4c906a83e19 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:17.951180 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4512ec-1a62-4ede-95eb-792872b33dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:17.952045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4512ec-1a62-4ede-95eb-792872b33dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:17.952323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4512ec-1a62-4ede-95eb-792872b33dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:17.952641 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4512ec-1a62-4ede-95eb-792872b33dbc admin admin] PATH_INFO: `/v3/projects/e6c03c3aff534d7d8b73d3199d925c09` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:17.953993 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4512ec-1a62-4ede-95eb-792872b33dbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6c03c3aff534d7d8b73d3199d925c09)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:17.956074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4512ec-1a62-4ede-95eb-792872b33dbc admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:18.009179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:18.010380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ea4512ec-1a62-4ede-95eb-792872b33dbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6c03c3aff534d7d8b73d3199d925c09', 'request_id': 'req-ea4512ec-1a62-4ede-95eb-792872b33dbc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:18.018750 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ea4512ec-1a62-4ede-95eb-792872b33dbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6c03c3aff534d7d8b73d3199d925c09', 'request_id': 'req-ea4512ec-1a62-4ede-95eb-792872b33dbc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:18.024550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ea4512ec-1a62-4ede-95eb-792872b33dbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6c03c3aff534d7d8b73d3199d925c09', 'request_id': 'req-ea4512ec-1a62-4ede-95eb-792872b33dbc'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:18.036997 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ea4512ec-1a62-4ede-95eb-792872b33dbc admin admin] The token cache is being invalidate because project e6c03c3aff534d7d8b73d3199d925c09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:41:18.037288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ea4512ec-1a62-4ede-95eb-792872b33dbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:18.037940 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ea4512ec-1a62-4ede-95eb-792872b33dbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:18.039535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1251/2517] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:41:17 2022] DELETE /identity/v3/projects/e6c03c3aff534d7d8b73d3199d925c09 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:41:18.044591 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3061428-c0d4-4990-9905-a3a699d861ba None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:18.044926 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3061428-c0d4-4990-9905-a3a699d861ba None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:18.045184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3061428-c0d4-4990-9905-a3a699d861ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:18.063912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-c3061428-c0d4-4990-9905-a3a699d861ba None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:18.065028 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-c3061428-c0d4-4990-9905-a3a699d861ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:18.123342 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1267/2518] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:18.131180 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf70995-fb1b-4e57-8ebe-2fde2b5ead90 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:18.131470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf70995-fb1b-4e57-8ebe-2fde2b5ead90 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:18.131748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf70995-fb1b-4e57-8ebe-2fde2b5ead90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:18.149844 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-5cf70995-fb1b-4e57-8ebe-2fde2b5ead90 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:18.151192 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5cf70995-fb1b-4e57-8ebe-2fde2b5ead90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:18.159133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1252/2519] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:18.163340 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3061428-c0d4-4990-9905-a3a699d861ba None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:18.173627 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3061428-c0d4-4990-9905-a3a699d861ba None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:18.178192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5dc864-53a6-4550-ba03-836f249a4590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:18.178841 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dc864-53a6-4550-ba03-836f249a4590 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:18.179158 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dc864-53a6-4550-ba03-836f249a4590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:18.179438 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dc864-53a6-4550-ba03-836f249a4590 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:18.180167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5dc864-53a6-4550-ba03-836f249a4590 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:41:18.180479 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5dc864-53a6-4550-ba03-836f249a4590 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:18.182063 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5dc864-53a6-4550-ba03-836f249a4590 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:18.194018 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1268/2520] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:41:18.198916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf70995-fb1b-4e57-8ebe-2fde2b5ead90 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:18.208261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf70995-fb1b-4e57-8ebe-2fde2b5ead90 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:18.213054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafec1c7-fa50-4489-8a76-134abd9e1e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:18.213706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafec1c7-fa50-4489-8a76-134abd9e1e1a admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:18.213990 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafec1c7-fa50-4489-8a76-134abd9e1e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:18.214263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafec1c7-fa50-4489-8a76-134abd9e1e1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:18.214981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafec1c7-fa50-4489-8a76-134abd9e1e1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:18.216878 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafec1c7-fa50-4489-8a76-134abd9e1e1a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:18.244122 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1253/2521] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:41:18 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:41:18.248872 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5dc864-53a6-4550-ba03-836f249a4590 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:18.259329 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5dc864-53a6-4550-ba03-836f249a4590 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:18.291856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:18.319261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efad0c1-ebdb-46be-861b-b5637debd62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:18.319933 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efad0c1-ebdb-46be-861b-b5637debd62f admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:18.320222 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efad0c1-ebdb-46be-861b-b5637debd62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:18.320510 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efad0c1-ebdb-46be-861b-b5637debd62f admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:18.321215 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efad0c1-ebdb-46be-861b-b5637debd62f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:18.323180 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efad0c1-ebdb-46be-861b-b5637debd62f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:18.343375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1269/2522] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:41:18 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:41:18.347475 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafec1c7-fa50-4489-8a76-134abd9e1e1a admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:18.358344 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafec1c7-fa50-4489-8a76-134abd9e1e1a admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:18.363087 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f009d-1279-4501-a854-14210528a3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:18.363739 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f009d-1279-4501-a854-14210528a3e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:18.364022 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f009d-1279-4501-a854-14210528a3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:18.364298 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f009d-1279-4501-a854-14210528a3e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:18.365021 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f009d-1279-4501-a854-14210528a3e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:18.366623 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f009d-1279-4501-a854-14210528a3e8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:18.377322 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1254/2523] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:18.381526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efad0c1-ebdb-46be-861b-b5637debd62f admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:18.391970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efad0c1-ebdb-46be-861b-b5637debd62f admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:18.396873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26720caf-6541-446a-994b-d7e900ed6adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:18.397521 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26720caf-6541-446a-994b-d7e900ed6adc admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:18.397803 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26720caf-6541-446a-994b-d7e900ed6adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:18.398078 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26720caf-6541-446a-994b-d7e900ed6adc admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:18.398830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26720caf-6541-446a-994b-d7e900ed6adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:18.400436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26720caf-6541-446a-994b-d7e900ed6adc admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:18.410632 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1270/2524] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:18.414746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f009d-1279-4501-a854-14210528a3e8 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:18.424962 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f009d-1279-4501-a854-14210528a3e8 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:18.430015 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf7316-2343-4095-aa3a-43b425f6c048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:18.430893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf7316-2343-4095-aa3a-43b425f6c048 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:18.431205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf7316-2343-4095-aa3a-43b425f6c048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:18.431484 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf7316-2343-4095-aa3a-43b425f6c048 admin admin] PATH_INFO: `/v3/projects/9dc06ac7a2c7403faa7b384ff27c97d7/users/f1dc40fef49a43eb8e54fea4723f53fd/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:18.445914 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf7316-2343-4095-aa3a-43b425f6c048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dc06ac7a2c7403faa7b384ff27c97d7, user_id=f1dc40fef49a43eb8e54fea4723f53fd, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:18.448693 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf7316-2343-4095-aa3a-43b425f6c048 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:18.462615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1255/2525] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:41:18 2022] PUT /identity/v3/projects/9dc06ac7a2c7403faa7b384ff27c97d7/users/f1dc40fef49a43eb8e54fea4723f53fd/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:18.473423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288f78e-5374-406c-89ca-666e0c2529d5 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:18.473715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288f78e-5374-406c-89ca-666e0c2529d5 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:18.473995 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288f78e-5374-406c-89ca-666e0c2529d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:18.501137 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-4288f78e-5374-406c-89ca-666e0c2529d5 None None] MFA Rules not processed for user `f1dc40fef49a43eb8e54fea4723f53fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:18.502288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4288f78e-5374-406c-89ca-666e0c2529d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:18.577752 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1271/2526] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:18.588844 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf7316-2343-4095-aa3a-43b425f6c048 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:18.592529 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-34bf7316-2343-4095-aa3a-43b425f6c048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:18.665362 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf7316-2343-4095-aa3a-43b425f6c048 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:18.670912 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2e83e-7e16-4f63-b353-1a8814a4afcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:18.671595 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2e83e-7e16-4f63-b353-1a8814a4afcc service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:18.671878 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2e83e-7e16-4f63-b353-1a8814a4afcc service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:18.672152 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2e83e-7e16-4f63-b353-1a8814a4afcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:18.682154 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2e83e-7e16-4f63-b353-1a8814a4afcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:18.684385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2e83e-7e16-4f63-b353-1a8814a4afcc service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:18.694168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1256/2527] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:41:18.912275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288f78e-5374-406c-89ca-666e0c2529d5 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:18.986516 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288f78e-5374-406c-89ca-666e0c2529d5 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:18.991364 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faaad72-de0c-4d8f-89cd-56615e01f3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:18.992256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faaad72-de0c-4d8f-89cd-56615e01f3ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:18.992596 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faaad72-de0c-4d8f-89cd-56615e01f3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:18.993077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faaad72-de0c-4d8f-89cd-56615e01f3ce admin admin] PATH_INFO: `/v3/users/f1dc40fef49a43eb8e54fea4723f53fd` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:18.994691 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faaad72-de0c-4d8f-89cd-56615e01f3ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1dc40fef49a43eb8e54fea4723f53fd)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:18.997260 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faaad72-de0c-4d8f-89cd-56615e01f3ce admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:19.051407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:19.053991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0faaad72-de0c-4d8f-89cd-56615e01f3ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1dc40fef49a43eb8e54fea4723f53fd'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:19.064477 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0faaad72-de0c-4d8f-89cd-56615e01f3ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1dc40fef49a43eb8e54fea4723f53fd'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:19.071942 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0faaad72-de0c-4d8f-89cd-56615e01f3ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1dc40fef49a43eb8e54fea4723f53fd'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:19.072635 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0faaad72-de0c-4d8f-89cd-56615e01f3ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1dc40fef49a43eb8e54fea4723f53fd'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:19.073252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0faaad72-de0c-4d8f-89cd-56615e01f3ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1dc40fef49a43eb8e54fea4723f53fd'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:19.090745 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1272/2528] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:41:18 2022] DELETE /identity/v3/users/f1dc40fef49a43eb8e54fea4723f53fd => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:41:19.095252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ecfc2-8c24-42d6-929f-3a8b4bcb4b54 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:19.095546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ecfc2-8c24-42d6-929f-3a8b4bcb4b54 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:19.095829 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ecfc2-8c24-42d6-929f-3a8b4bcb4b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:19.115312 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f81ecfc2-8c24-42d6-929f-3a8b4bcb4b54 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:19.116536 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f81ecfc2-8c24-42d6-929f-3a8b4bcb4b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:19.165602 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:19.177246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1257/2529] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:19.187612 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faaad72-de0c-4d8f-89cd-56615e01f3ce admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:19.189002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-0faaad72-de0c-4d8f-89cd-56615e01f3ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:19.245824 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faaad72-de0c-4d8f-89cd-56615e01f3ce admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:19.249511 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b0915-20b0-4a1e-9169-adb920eaa2af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:19.250161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b0915-20b0-4a1e-9169-adb920eaa2af service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:19.250448 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b0915-20b0-4a1e-9169-adb920eaa2af service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:19.250726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b0915-20b0-4a1e-9169-adb920eaa2af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:19.259721 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b0915-20b0-4a1e-9169-adb920eaa2af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:19.261146 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b0915-20b0-4a1e-9169-adb920eaa2af service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:19.270952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1273/2530] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:41:19.542524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ecfc2-8c24-42d6-929f-3a8b4bcb4b54 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:19.544616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f81ecfc2-8c24-42d6-929f-3a8b4bcb4b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:19.562498 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ecfc2-8c24-42d6-929f-3a8b4bcb4b54 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:19.570850 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced0533-51d0-4398-af3d-d623f3702609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:19.573594 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced0533-51d0-4398-af3d-d623f3702609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:19.573594 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced0533-51d0-4398-af3d-d623f3702609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:19.573836 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced0533-51d0-4398-af3d-d623f3702609 admin admin] PATH_INFO: `/v3/projects/9dc06ac7a2c7403faa7b384ff27c97d7` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:19.575308 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced0533-51d0-4398-af3d-d623f3702609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dc06ac7a2c7403faa7b384ff27c97d7)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:19.577208 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced0533-51d0-4398-af3d-d623f3702609 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:19.630539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4ced0533-51d0-4398-af3d-d623f3702609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dc06ac7a2c7403faa7b384ff27c97d7', 'request_id': 'req-4ced0533-51d0-4398-af3d-d623f3702609'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:19.638973 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4ced0533-51d0-4398-af3d-d623f3702609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dc06ac7a2c7403faa7b384ff27c97d7', 'request_id': 'req-4ced0533-51d0-4398-af3d-d623f3702609'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:19.645513 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4ced0533-51d0-4398-af3d-d623f3702609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dc06ac7a2c7403faa7b384ff27c97d7', 'request_id': 'req-4ced0533-51d0-4398-af3d-d623f3702609'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:19.660321 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4ced0533-51d0-4398-af3d-d623f3702609 admin admin] The token cache is being invalidate because project 9dc06ac7a2c7403faa7b384ff27c97d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:41:19.660622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4ced0533-51d0-4398-af3d-d623f3702609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:19.661404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4ced0533-51d0-4398-af3d-d623f3702609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:19.663002 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1258/2531] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:41:19 2022] DELETE /identity/v3/projects/9dc06ac7a2c7403faa7b384ff27c97d7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:19.668198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0e34a-62d6-4c62-8c80-0c9738cf4381 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:19.668719 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0e34a-62d6-4c62-8c80-0c9738cf4381 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:19.668957 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0e34a-62d6-4c62-8c80-0c9738cf4381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:19.687360 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-3ce0e34a-62d6-4c62-8c80-0c9738cf4381 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:19.688617 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3ce0e34a-62d6-4c62-8c80-0c9738cf4381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:19.749722 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1274/2532] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:19.758147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d205c71-a157-44be-9729-f47c31bb5652 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:19.758147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d205c71-a157-44be-9729-f47c31bb5652 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:19.758147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d205c71-a157-44be-9729-f47c31bb5652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:19.778114 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8d205c71-a157-44be-9729-f47c31bb5652 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:19.779382 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8d205c71-a157-44be-9729-f47c31bb5652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:19.787867 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1259/2533] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:19.792293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0e34a-62d6-4c62-8c80-0c9738cf4381 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:19.803092 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0e34a-62d6-4c62-8c80-0c9738cf4381 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:19.817985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caad45d-c2f0-47ab-bec5-70557f0734bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:19.819199 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caad45d-c2f0-47ab-bec5-70557f0734bc admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:19.819499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caad45d-c2f0-47ab-bec5-70557f0734bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:19.819781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caad45d-c2f0-47ab-bec5-70557f0734bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:19.820594 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caad45d-c2f0-47ab-bec5-70557f0734bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:41:19.820920 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caad45d-c2f0-47ab-bec5-70557f0734bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:19.822543 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caad45d-c2f0-47ab-bec5-70557f0734bc admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:19.838041 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1275/2534] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:41:19.843609 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d205c71-a157-44be-9729-f47c31bb5652 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:19.853580 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d205c71-a157-44be-9729-f47c31bb5652 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:19.858929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6af83-7dec-405f-b594-8935869a3883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:19.859617 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6af83-7dec-405f-b594-8935869a3883 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:19.859957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6af83-7dec-405f-b594-8935869a3883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:19.860247 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6af83-7dec-405f-b594-8935869a3883 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:19.860988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6af83-7dec-405f-b594-8935869a3883 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:19.862896 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6af83-7dec-405f-b594-8935869a3883 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:19.893293 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1260/2535] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:41:19 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:41:19.897636 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caad45d-c2f0-47ab-bec5-70557f0734bc admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:19.908244 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caad45d-c2f0-47ab-bec5-70557f0734bc admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:19.961610 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:19.971992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf6a39-a665-4b15-9123-00fa9fbb4764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:19.972760 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf6a39-a665-4b15-9123-00fa9fbb4764 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:19.972953 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf6a39-a665-4b15-9123-00fa9fbb4764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:19.973516 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf6a39-a665-4b15-9123-00fa9fbb4764 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:19.974903 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf6a39-a665-4b15-9123-00fa9fbb4764 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:19.977561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf6a39-a665-4b15-9123-00fa9fbb4764 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:20.005909 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1276/2536] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:41:19 2022] POST /identity/v3/users => generated 282 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:41:20.010075 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6af83-7dec-405f-b594-8935869a3883 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:20.021450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6af83-7dec-405f-b594-8935869a3883 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:20.026351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ce8a8-0d0d-4eba-b331-ed6f6829a6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:20.027072 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ce8a8-0d0d-4eba-b331-ed6f6829a6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:20.027357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ce8a8-0d0d-4eba-b331-ed6f6829a6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:20.028245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ce8a8-0d0d-4eba-b331-ed6f6829a6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:20.028373 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ce8a8-0d0d-4eba-b331-ed6f6829a6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:20.029967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ce8a8-0d0d-4eba-b331-ed6f6829a6e0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:20.040783 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1261/2537] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:20.045291 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf6a39-a665-4b15-9123-00fa9fbb4764 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:20.055730 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf6a39-a665-4b15-9123-00fa9fbb4764 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:20.060991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06b68c-87ca-4844-b7c8-093bf28d4c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:20.061671 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06b68c-87ca-4844-b7c8-093bf28d4c4c admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:20.061979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06b68c-87ca-4844-b7c8-093bf28d4c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:20.062258 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06b68c-87ca-4844-b7c8-093bf28d4c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:20.063026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06b68c-87ca-4844-b7c8-093bf28d4c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:20.064631 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06b68c-87ca-4844-b7c8-093bf28d4c4c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:20.076107 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1277/2538] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:20.081051 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ce8a8-0d0d-4eba-b331-ed6f6829a6e0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:20.091224 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ce8a8-0d0d-4eba-b331-ed6f6829a6e0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:20.096190 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4add1e-ae03-4da5-b977-d60916b96c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:20.097080 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4add1e-ae03-4da5-b977-d60916b96c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:20.097363 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4add1e-ae03-4da5-b977-d60916b96c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:20.097638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4add1e-ae03-4da5-b977-d60916b96c57 admin admin] PATH_INFO: `/v3/projects/d9a83d30ebfc49d39ceb077467c74d87/users/f55622da0f794d7abe567d9dbbe8c4a6/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:20.113472 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4add1e-ae03-4da5-b977-d60916b96c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9a83d30ebfc49d39ceb077467c74d87, user_id=f55622da0f794d7abe567d9dbbe8c4a6, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:20.116195 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4add1e-ae03-4da5-b977-d60916b96c57 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:20.130660 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1262/2539] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:41:20 2022] PUT /identity/v3/projects/d9a83d30ebfc49d39ceb077467c74d87/users/f55622da0f794d7abe567d9dbbe8c4a6/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:20.142152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62ce93-2367-471a-b8c8-b87044f8b851 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:20.142457 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62ce93-2367-471a-b8c8-b87044f8b851 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:20.142705 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62ce93-2367-471a-b8c8-b87044f8b851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:20.172065 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-6c62ce93-2367-471a-b8c8-b87044f8b851 None None] MFA Rules not processed for user `f55622da0f794d7abe567d9dbbe8c4a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:20.173231 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6c62ce93-2367-471a-b8c8-b87044f8b851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:20.239046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1278/2540] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:20 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:20.243586 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4add1e-ae03-4da5-b977-d60916b96c57 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:20.306846 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4add1e-ae03-4da5-b977-d60916b96c57 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:20.313274 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d2be5-25b0-4050-9b8b-b1a03d1dd539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:20.313950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d2be5-25b0-4050-9b8b-b1a03d1dd539 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:20.314238 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d2be5-25b0-4050-9b8b-b1a03d1dd539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:20.314520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d2be5-25b0-4050-9b8b-b1a03d1dd539 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:20.315306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d2be5-25b0-4050-9b8b-b1a03d1dd539 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:20.317210 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d2be5-25b0-4050-9b8b-b1a03d1dd539 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:20.344654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1263/2541] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:41:20 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:41:20.348712 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62ce93-2367-471a-b8c8-b87044f8b851 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:20.360349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62ce93-2367-471a-b8c8-b87044f8b851 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:20.421792 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c817bb-f6e0-4efc-905c-abbd68ed4856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:20.422568 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c817bb-f6e0-4efc-905c-abbd68ed4856 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:20.422801 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c817bb-f6e0-4efc-905c-abbd68ed4856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:20.423124 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c817bb-f6e0-4efc-905c-abbd68ed4856 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:20.423885 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c817bb-f6e0-4efc-905c-abbd68ed4856 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:20.426329 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c817bb-f6e0-4efc-905c-abbd68ed4856 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:20.447191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1279/2542] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:41:20 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:41:20.451356 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d2be5-25b0-4050-9b8b-b1a03d1dd539 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:20.456851 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:20.462763 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d2be5-25b0-4050-9b8b-b1a03d1dd539 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:20.468902 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278bae3-cb86-4e33-af9e-77d177aca593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:20.469741 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278bae3-cb86-4e33-af9e-77d177aca593 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:20.470251 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278bae3-cb86-4e33-af9e-77d177aca593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:20.470512 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278bae3-cb86-4e33-af9e-77d177aca593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:20.471287 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278bae3-cb86-4e33-af9e-77d177aca593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:20.472981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278bae3-cb86-4e33-af9e-77d177aca593 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:20.484363 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1264/2543] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:20.488599 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c817bb-f6e0-4efc-905c-abbd68ed4856 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:20.499532 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c817bb-f6e0-4efc-905c-abbd68ed4856 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:20.504366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f7db5-55b3-4350-a47d-693dda2443f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:20.505251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f7db5-55b3-4350-a47d-693dda2443f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:20.505529 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f7db5-55b3-4350-a47d-693dda2443f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:20.505806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f7db5-55b3-4350-a47d-693dda2443f8 admin admin] PATH_INFO: `/v3/projects/0239c0e06cb945b7a4a6e42ffe7c1f78/users/9831464a17e948f9b96f8b7af86c689f/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:20.521284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011f7db5-55b3-4350-a47d-693dda2443f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0239c0e06cb945b7a4a6e42ffe7c1f78, user_id=9831464a17e948f9b96f8b7af86c689f, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:20.523985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011f7db5-55b3-4350-a47d-693dda2443f8 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:20.536990 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1280/2544] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:41:20 2022] PUT /identity/v3/projects/0239c0e06cb945b7a4a6e42ffe7c1f78/users/9831464a17e948f9b96f8b7af86c689f/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:20.541129 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278bae3-cb86-4e33-af9e-77d177aca593 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:20.552140 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278bae3-cb86-4e33-af9e-77d177aca593 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:20.609992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28698e17-5f02-4039-9396-29da15626081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:20.610674 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28698e17-5f02-4039-9396-29da15626081 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:20.610989 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28698e17-5f02-4039-9396-29da15626081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:20.611269 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28698e17-5f02-4039-9396-29da15626081 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:20.611996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28698e17-5f02-4039-9396-29da15626081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:20.613601 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28698e17-5f02-4039-9396-29da15626081 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:20.627300 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1265/2545] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:20.632015 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f7db5-55b3-4350-a47d-693dda2443f8 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:20.643110 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f7db5-55b3-4350-a47d-693dda2443f8 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:20.648237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a55052-780e-4ef9-a46d-80073c0d051c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:20.649136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a55052-780e-4ef9-a46d-80073c0d051c admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:20.649440 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a55052-780e-4ef9-a46d-80073c0d051c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:20.649716 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a55052-780e-4ef9-a46d-80073c0d051c admin admin] PATH_INFO: `/v3/projects/0239c0e06cb945b7a4a6e42ffe7c1f78/users/9831464a17e948f9b96f8b7af86c689f/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:20.651947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a55052-780e-4ef9-a46d-80073c0d051c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0239c0e06cb945b7a4a6e42ffe7c1f78, user_id=9831464a17e948f9b96f8b7af86c689f, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:20.654597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a55052-780e-4ef9-a46d-80073c0d051c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:20.666365 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1281/2546] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:41:20 2022] PUT /identity/v3/projects/0239c0e06cb945b7a4a6e42ffe7c1f78/users/9831464a17e948f9b96f8b7af86c689f/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:41:20.679087 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77c712-9988-48aa-8293-a3771cc30bfa None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:20.679298 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77c712-9988-48aa-8293-a3771cc30bfa None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:20.679538 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77c712-9988-48aa-8293-a3771cc30bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:20.709840 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-3b77c712-9988-48aa-8293-a3771cc30bfa None None] MFA Rules not processed for user `9831464a17e948f9b96f8b7af86c689f`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:20.711122 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3b77c712-9988-48aa-8293-a3771cc30bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:20.794817 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1266/2547] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:20 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:20.806733 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a55052-780e-4ef9-a46d-80073c0d051c admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:20.808864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-75a55052-780e-4ef9-a46d-80073c0d051c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:20.831662 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:20.907203 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a55052-780e-4ef9-a46d-80073c0d051c admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:20.911160 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4162c-4731-4258-9df8-01d92a385384 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:20.911822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4162c-4731-4258-9df8-01d92a385384 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:20.912105 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4162c-4731-4258-9df8-01d92a385384 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:20.912383 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4162c-4731-4258-9df8-01d92a385384 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:20.923629 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4162c-4731-4258-9df8-01d92a385384 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:20.925246 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4162c-4731-4258-9df8-01d92a385384 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:20.935940 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1282/2548] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:20 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:41:21.012960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77c712-9988-48aa-8293-a3771cc30bfa None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:21.023436 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77c712-9988-48aa-8293-a3771cc30bfa None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:21.028332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce0c97-44c6-44e1-aa3a-f51f6cc2daf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:21.028993 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce0c97-44c6-44e1-aa3a-f51f6cc2daf0 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:21.029394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce0c97-44c6-44e1-aa3a-f51f6cc2daf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:21.029524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce0c97-44c6-44e1-aa3a-f51f6cc2daf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:21.098087 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce0c97-44c6-44e1-aa3a-f51f6cc2daf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:21.099645 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce0c97-44c6-44e1-aa3a-f51f6cc2daf0 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:21.109944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1267/2549] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:21 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:41:30.931116 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=29 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:31.105665 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:31.436703 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4162c-4731-4258-9df8-01d92a385384 service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:31.497485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4162c-4731-4258-9df8-01d92a385384 service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:31.502007 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bc50d-1ade-417b-8011-c18045a8009f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:31.502878 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bc50d-1ade-417b-8011-c18045a8009f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:31.503190 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bc50d-1ade-417b-8011-c18045a8009f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:31.503466 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bc50d-1ade-417b-8011-c18045a8009f admin admin] PATH_INFO: `/v3/users/f55622da0f794d7abe567d9dbbe8c4a6` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:31.504631 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bc50d-1ade-417b-8011-c18045a8009f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f55622da0f794d7abe567d9dbbe8c4a6)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:31.506543 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bc50d-1ade-417b-8011-c18045a8009f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:31.552742 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-572bc50d-1ade-417b-8011-c18045a8009f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f55622da0f794d7abe567d9dbbe8c4a6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:31.559939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-572bc50d-1ade-417b-8011-c18045a8009f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f55622da0f794d7abe567d9dbbe8c4a6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:31.566792 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-572bc50d-1ade-417b-8011-c18045a8009f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f55622da0f794d7abe567d9dbbe8c4a6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:31.567450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-572bc50d-1ade-417b-8011-c18045a8009f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f55622da0f794d7abe567d9dbbe8c4a6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:31.568056 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-572bc50d-1ade-417b-8011-c18045a8009f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f55622da0f794d7abe567d9dbbe8c4a6'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:31.585227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1283/2550] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:41:31 2022] DELETE /identity/v3/users/f55622da0f794d7abe567d9dbbe8c4a6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:31.590198 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02765007-c7d6-4df3-b0fe-874908d68859 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:31.590543 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02765007-c7d6-4df3-b0fe-874908d68859 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:31.590712 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02765007-c7d6-4df3-b0fe-874908d68859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:31.733509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-02765007-c7d6-4df3-b0fe-874908d68859 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:31.734750 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-02765007-c7d6-4df3-b0fe-874908d68859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:31.795908 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1268/2551] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:31.805566 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bc50d-1ade-417b-8011-c18045a8009f admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:31.806961 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-572bc50d-1ade-417b-8011-c18045a8009f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:31.865335 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bc50d-1ade-417b-8011-c18045a8009f admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:31.868937 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244d97e-9ba9-4018-8e9f-b4e460df2b27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:31.869579 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244d97e-9ba9-4018-8e9f-b4e460df2b27 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:31.869860 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244d97e-9ba9-4018-8e9f-b4e460df2b27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:31.870134 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244d97e-9ba9-4018-8e9f-b4e460df2b27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:31.879949 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244d97e-9ba9-4018-8e9f-b4e460df2b27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:31.881366 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244d97e-9ba9-4018-8e9f-b4e460df2b27 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:31.890847 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1284/2552] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:41:32.182279 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02765007-c7d6-4df3-b0fe-874908d68859 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:32.183953 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-02765007-c7d6-4df3-b0fe-874908d68859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:32.196247 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02765007-c7d6-4df3-b0fe-874908d68859 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:32.201385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d21f09-72ab-4215-b6d4-b9beea847f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:32.202259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d21f09-72ab-4215-b6d4-b9beea847f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:32.202539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d21f09-72ab-4215-b6d4-b9beea847f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:32.202861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d21f09-72ab-4215-b6d4-b9beea847f54 admin admin] PATH_INFO: `/v3/projects/d9a83d30ebfc49d39ceb077467c74d87` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:32.204327 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d21f09-72ab-4215-b6d4-b9beea847f54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9a83d30ebfc49d39ceb077467c74d87)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:32.206252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d21f09-72ab-4215-b6d4-b9beea847f54 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:32.257170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-49d21f09-72ab-4215-b6d4-b9beea847f54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9a83d30ebfc49d39ceb077467c74d87', 'request_id': 'req-49d21f09-72ab-4215-b6d4-b9beea847f54'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:32.263564 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-49d21f09-72ab-4215-b6d4-b9beea847f54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9a83d30ebfc49d39ceb077467c74d87', 'request_id': 'req-49d21f09-72ab-4215-b6d4-b9beea847f54'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:32.272262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-49d21f09-72ab-4215-b6d4-b9beea847f54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9a83d30ebfc49d39ceb077467c74d87', 'request_id': 'req-49d21f09-72ab-4215-b6d4-b9beea847f54'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:32.289137 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-49d21f09-72ab-4215-b6d4-b9beea847f54 admin admin] The token cache is being invalidate because project d9a83d30ebfc49d39ceb077467c74d87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:41:32.289436 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-49d21f09-72ab-4215-b6d4-b9beea847f54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:32.290075 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-49d21f09-72ab-4215-b6d4-b9beea847f54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:32.291786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1269/2553] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:41:32 2022] DELETE /identity/v3/projects/d9a83d30ebfc49d39ceb077467c74d87 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:32.296399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244d97e-9ba9-4018-8e9f-b4e460df2b27 service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:32.306411 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244d97e-9ba9-4018-8e9f-b4e460df2b27 service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:32.325985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:32.371405 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479ef22-7be2-4e84-b6b5-cd8d43d71b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:32.372271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479ef22-7be2-4e84-b6b5-cd8d43d71b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:32.372552 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479ef22-7be2-4e84-b6b5-cd8d43d71b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:32.372821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479ef22-7be2-4e84-b6b5-cd8d43d71b79 admin admin] PATH_INFO: `/v3/users/9831464a17e948f9b96f8b7af86c689f` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:32.374046 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8479ef22-7be2-4e84-b6b5-cd8d43d71b79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9831464a17e948f9b96f8b7af86c689f)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:32.376027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8479ef22-7be2-4e84-b6b5-cd8d43d71b79 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:32.421981 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8479ef22-7be2-4e84-b6b5-cd8d43d71b79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9831464a17e948f9b96f8b7af86c689f'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:32.429499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8479ef22-7be2-4e84-b6b5-cd8d43d71b79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9831464a17e948f9b96f8b7af86c689f'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:32.437372 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8479ef22-7be2-4e84-b6b5-cd8d43d71b79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9831464a17e948f9b96f8b7af86c689f'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:32.438079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8479ef22-7be2-4e84-b6b5-cd8d43d71b79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9831464a17e948f9b96f8b7af86c689f'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:32.438718 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8479ef22-7be2-4e84-b6b5-cd8d43d71b79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9831464a17e948f9b96f8b7af86c689f'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:32.457703 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1285/2554] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:41:32 2022] DELETE /identity/v3/users/9831464a17e948f9b96f8b7af86c689f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:32.735520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d21f09-72ab-4215-b6d4-b9beea847f54 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:32.736959 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-49d21f09-72ab-4215-b6d4-b9beea847f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:32.751625 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d21f09-72ab-4215-b6d4-b9beea847f54 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:32.829575 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a9f3b-fdcc-4565-bb27-7eaabf0f7291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:32.830442 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a9f3b-fdcc-4565-bb27-7eaabf0f7291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:32.830725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a9f3b-fdcc-4565-bb27-7eaabf0f7291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:32.831024 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a9f3b-fdcc-4565-bb27-7eaabf0f7291 admin admin] PATH_INFO: `/v3/projects/0239c0e06cb945b7a4a6e42ffe7c1f78` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:32.832233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a9f3b-fdcc-4565-bb27-7eaabf0f7291 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0239c0e06cb945b7a4a6e42ffe7c1f78)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:32.834159 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a9f3b-fdcc-4565-bb27-7eaabf0f7291 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:32.865626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:32.884656 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aa8a9f3b-fdcc-4565-bb27-7eaabf0f7291 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0239c0e06cb945b7a4a6e42ffe7c1f78', 'request_id': 'req-aa8a9f3b-fdcc-4565-bb27-7eaabf0f7291'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:32.891604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aa8a9f3b-fdcc-4565-bb27-7eaabf0f7291 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0239c0e06cb945b7a4a6e42ffe7c1f78', 'request_id': 'req-aa8a9f3b-fdcc-4565-bb27-7eaabf0f7291'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:32.897524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aa8a9f3b-fdcc-4565-bb27-7eaabf0f7291 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0239c0e06cb945b7a4a6e42ffe7c1f78', 'request_id': 'req-aa8a9f3b-fdcc-4565-bb27-7eaabf0f7291'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:32.910315 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aa8a9f3b-fdcc-4565-bb27-7eaabf0f7291 admin admin] The token cache is being invalidate because project 0239c0e06cb945b7a4a6e42ffe7c1f78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:41:32.910606 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aa8a9f3b-fdcc-4565-bb27-7eaabf0f7291 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:32.911254 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aa8a9f3b-fdcc-4565-bb27-7eaabf0f7291 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:32.912788 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1270/2555] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:41:32 2022] DELETE /identity/v3/projects/0239c0e06cb945b7a4a6e42ffe7c1f78 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:32.917640 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b77e1a-62f3-4ae6-b5dd-de498ed36647 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:32.917930 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b77e1a-62f3-4ae6-b5dd-de498ed36647 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:32.918208 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b77e1a-62f3-4ae6-b5dd-de498ed36647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:32.936599 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-05b77e1a-62f3-4ae6-b5dd-de498ed36647 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:32.937816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-05b77e1a-62f3-4ae6-b5dd-de498ed36647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:32.996500 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1286/2556] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:33.004214 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ca19b-5e78-4141-899f-fe15d3a1c8b6 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:33.004498 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ca19b-5e78-4141-899f-fe15d3a1c8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:33.004770 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ca19b-5e78-4141-899f-fe15d3a1c8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:33.023135 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-840ca19b-5e78-4141-899f-fe15d3a1c8b6 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:33.024314 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-840ca19b-5e78-4141-899f-fe15d3a1c8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:33.032060 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1271/2557] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:33.036154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b77e1a-62f3-4ae6-b5dd-de498ed36647 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:33.045697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b77e1a-62f3-4ae6-b5dd-de498ed36647 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:33.050243 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f9d95-5d25-4bdc-99f4-831784303bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:33.050883 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f9d95-5d25-4bdc-99f4-831784303bac admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:33.051195 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f9d95-5d25-4bdc-99f4-831784303bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:33.051467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f9d95-5d25-4bdc-99f4-831784303bac admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:33.052188 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f9d95-5d25-4bdc-99f4-831784303bac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:41:33.052522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f9d95-5d25-4bdc-99f4-831784303bac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:33.054096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f9d95-5d25-4bdc-99f4-831784303bac admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:33.065798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1287/2558] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:41:33.070102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ca19b-5e78-4141-899f-fe15d3a1c8b6 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:33.079836 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ca19b-5e78-4141-899f-fe15d3a1c8b6 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:33.084498 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe62cce-f430-4b93-a3b1-0a8574740b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:33.085137 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe62cce-f430-4b93-a3b1-0a8574740b82 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:33.085418 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe62cce-f430-4b93-a3b1-0a8574740b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:33.085689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe62cce-f430-4b93-a3b1-0a8574740b82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:33.086368 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe62cce-f430-4b93-a3b1-0a8574740b82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:33.088277 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe62cce-f430-4b93-a3b1-0a8574740b82 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:33.114233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1272/2559] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:41:33 2022] POST /identity/v3/projects => generated 399 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:41:33.118452 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f9d95-5d25-4bdc-99f4-831784303bac admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:33.128869 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f9d95-5d25-4bdc-99f4-831784303bac admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:33.154129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:33.185086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c68ad-6321-4362-8575-eaeed4fd85e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:33.185733 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c68ad-6321-4362-8575-eaeed4fd85e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:33.186014 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c68ad-6321-4362-8575-eaeed4fd85e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:33.186288 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c68ad-6321-4362-8575-eaeed4fd85e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:33.187008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c68ad-6321-4362-8575-eaeed4fd85e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:33.188911 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c68ad-6321-4362-8575-eaeed4fd85e8 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:33.220350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1288/2560] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:41:33 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:41:33.224482 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe62cce-f430-4b93-a3b1-0a8574740b82 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:33.234803 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe62cce-f430-4b93-a3b1-0a8574740b82 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:33.239453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a36eaa-feed-4bc5-8503-f70623d58f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:33.240106 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a36eaa-feed-4bc5-8503-f70623d58f3c admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:33.240389 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a36eaa-feed-4bc5-8503-f70623d58f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:33.240663 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a36eaa-feed-4bc5-8503-f70623d58f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:33.241395 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a36eaa-feed-4bc5-8503-f70623d58f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:33.242996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a36eaa-feed-4bc5-8503-f70623d58f3c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:33.253266 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1273/2561] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:33.257156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c68ad-6321-4362-8575-eaeed4fd85e8 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:33.267771 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c68ad-6321-4362-8575-eaeed4fd85e8 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:33.273041 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fc9c1-b7e2-4c21-8752-f6d4c036e3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:33.273691 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fc9c1-b7e2-4c21-8752-f6d4c036e3ba admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:33.273994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fc9c1-b7e2-4c21-8752-f6d4c036e3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:33.274252 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fc9c1-b7e2-4c21-8752-f6d4c036e3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:33.275009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fc9c1-b7e2-4c21-8752-f6d4c036e3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:33.276596 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fc9c1-b7e2-4c21-8752-f6d4c036e3ba admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:33.287675 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1289/2562] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:33.291772 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a36eaa-feed-4bc5-8503-f70623d58f3c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:33.301724 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a36eaa-feed-4bc5-8503-f70623d58f3c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:33.306361 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bede4-2a0a-4c65-ba9c-7f82eb9ef179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:33.307273 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bede4-2a0a-4c65-ba9c-7f82eb9ef179 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:33.307553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bede4-2a0a-4c65-ba9c-7f82eb9ef179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:33.307829 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bede4-2a0a-4c65-ba9c-7f82eb9ef179 admin admin] PATH_INFO: `/v3/projects/90e539bf7c29493a80a26b492c0c3006/users/7d6f8fdcc3524bafa847b9d8c12e8c12/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:33.322133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bede4-2a0a-4c65-ba9c-7f82eb9ef179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90e539bf7c29493a80a26b492c0c3006, user_id=7d6f8fdcc3524bafa847b9d8c12e8c12, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:33.325324 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bede4-2a0a-4c65-ba9c-7f82eb9ef179 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:33.338007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1274/2563] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:41:33 2022] PUT /identity/v3/projects/90e539bf7c29493a80a26b492c0c3006/users/7d6f8fdcc3524bafa847b9d8c12e8c12/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:33.348551 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e2aa4-f59e-4b98-85af-3b9ec80eb1a7 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:33.349042 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e2aa4-f59e-4b98-85af-3b9ec80eb1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:33.349271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e2aa4-f59e-4b98-85af-3b9ec80eb1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:33.376896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-1c3e2aa4-f59e-4b98-85af-3b9ec80eb1a7 None None] MFA Rules not processed for user `7d6f8fdcc3524bafa847b9d8c12e8c12`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:33.378097 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-1c3e2aa4-f59e-4b98-85af-3b9ec80eb1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:33.448093 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1290/2564] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:33.453093 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bede4-2a0a-4c65-ba9c-7f82eb9ef179 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:33.516644 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bede4-2a0a-4c65-ba9c-7f82eb9ef179 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:33.521622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ddee5-6f1f-453c-b2d0-dbbd4d882bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:33.522280 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ddee5-6f1f-453c-b2d0-dbbd4d882bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:33.522560 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ddee5-6f1f-453c-b2d0-dbbd4d882bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:33.522829 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ddee5-6f1f-453c-b2d0-dbbd4d882bf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:33.523561 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ddee5-6f1f-453c-b2d0-dbbd4d882bf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:33.525452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ddee5-6f1f-453c-b2d0-dbbd4d882bf0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:33.553960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1275/2565] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:41:33 2022] POST /identity/v3/projects => generated 401 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:41:33.558661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e2aa4-f59e-4b98-85af-3b9ec80eb1a7 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:33.570007 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e2aa4-f59e-4b98-85af-3b9ec80eb1a7 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:33.630063 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17072443-195d-472d-9262-a65ed602b972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:33.630700 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17072443-195d-472d-9262-a65ed602b972 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:33.631013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17072443-195d-472d-9262-a65ed602b972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:33.631296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17072443-195d-472d-9262-a65ed602b972 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:33.632019 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17072443-195d-472d-9262-a65ed602b972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:33.633967 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17072443-195d-472d-9262-a65ed602b972 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:33.655689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1291/2566] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:41:33 2022] POST /identity/v3/users => generated 307 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:41:33.659902 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ddee5-6f1f-453c-b2d0-dbbd4d882bf0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:33.671125 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ddee5-6f1f-453c-b2d0-dbbd4d882bf0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:33.675993 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd2653-fe38-4125-906c-c9700fc5889e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:33.676654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd2653-fe38-4125-906c-c9700fc5889e admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:33.676940 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd2653-fe38-4125-906c-c9700fc5889e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:33.677217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd2653-fe38-4125-906c-c9700fc5889e admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:33.677947 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd2653-fe38-4125-906c-c9700fc5889e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:33.679598 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd2653-fe38-4125-906c-c9700fc5889e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:33.692270 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1276/2567] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:41:33.696497 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17072443-195d-472d-9262-a65ed602b972 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:33.707229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17072443-195d-472d-9262-a65ed602b972 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:33.711796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc941ed8-97b8-467e-9476-9681632f5cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:33.712672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc941ed8-97b8-467e-9476-9681632f5cc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:33.712954 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc941ed8-97b8-467e-9476-9681632f5cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:33.713229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc941ed8-97b8-467e-9476-9681632f5cc4 admin admin] PATH_INFO: `/v3/projects/1c65bd03ee674156bf390dacaf21f43b/users/282ff58f93774ddaac047a729fdb0977/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:33.727604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc941ed8-97b8-467e-9476-9681632f5cc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c65bd03ee674156bf390dacaf21f43b, user_id=282ff58f93774ddaac047a729fdb0977, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:33.730240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc941ed8-97b8-467e-9476-9681632f5cc4 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:33.744727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1292/2568] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:41:33 2022] PUT /identity/v3/projects/1c65bd03ee674156bf390dacaf21f43b/users/282ff58f93774ddaac047a729fdb0977/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:33.749370 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd2653-fe38-4125-906c-c9700fc5889e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:33.759601 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd2653-fe38-4125-906c-c9700fc5889e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:33.782124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:33.820693 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66890040-ede0-4db2-8447-832f881cde9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:33.821365 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66890040-ede0-4db2-8447-832f881cde9d admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:33.821649 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66890040-ede0-4db2-8447-832f881cde9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:33.821989 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66890040-ede0-4db2-8447-832f881cde9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:33.822792 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66890040-ede0-4db2-8447-832f881cde9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:33.825497 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66890040-ede0-4db2-8447-832f881cde9d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:33.836121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1277/2569] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:33.840441 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc941ed8-97b8-467e-9476-9681632f5cc4 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:33.845921 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:33.851831 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc941ed8-97b8-467e-9476-9681632f5cc4 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:33.857494 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53220d8e-3dae-46bf-8148-9d0da1b5fbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:33.858398 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53220d8e-3dae-46bf-8148-9d0da1b5fbfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:33.858684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53220d8e-3dae-46bf-8148-9d0da1b5fbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:33.858991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53220d8e-3dae-46bf-8148-9d0da1b5fbfe admin admin] PATH_INFO: `/v3/projects/1c65bd03ee674156bf390dacaf21f43b/users/282ff58f93774ddaac047a729fdb0977/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:33.861110 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53220d8e-3dae-46bf-8148-9d0da1b5fbfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c65bd03ee674156bf390dacaf21f43b, user_id=282ff58f93774ddaac047a729fdb0977, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:33.863729 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53220d8e-3dae-46bf-8148-9d0da1b5fbfe admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:33.875067 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1293/2570] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:41:33 2022] PUT /identity/v3/projects/1c65bd03ee674156bf390dacaf21f43b/users/282ff58f93774ddaac047a729fdb0977/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:33.886379 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d1f54-b711-4364-84f4-7333b05670f8 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:33.886605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d1f54-b711-4364-84f4-7333b05670f8 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:33.886889 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d1f54-b711-4364-84f4-7333b05670f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:33.912704 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a92d1f54-b711-4364-84f4-7333b05670f8 None None] MFA Rules not processed for user `282ff58f93774ddaac047a729fdb0977`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:33.913913 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a92d1f54-b711-4364-84f4-7333b05670f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:33.983588 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1278/2571] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:33 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:33.995199 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53220d8e-3dae-46bf-8148-9d0da1b5fbfe admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:33.999573 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-53220d8e-3dae-46bf-8148-9d0da1b5fbfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:34.059332 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53220d8e-3dae-46bf-8148-9d0da1b5fbfe admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:34.063015 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbd47c-c9d4-4eac-9fe0-fcd4aef4bd51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:34.063672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbd47c-c9d4-4eac-9fe0-fcd4aef4bd51 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:34.063952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbd47c-c9d4-4eac-9fe0-fcd4aef4bd51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:34.064228 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbd47c-c9d4-4eac-9fe0-fcd4aef4bd51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:34.073413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbd47c-c9d4-4eac-9fe0-fcd4aef4bd51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:34.074910 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbd47c-c9d4-4eac-9fe0-fcd4aef4bd51 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:34.084587 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1294/2572] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:33 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:41:34.157243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d1f54-b711-4364-84f4-7333b05670f8 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:34.168136 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d1f54-b711-4364-84f4-7333b05670f8 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:34.172660 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b22d3-1764-4fa1-8dd5-612e43a92461 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:34.173312 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b22d3-1764-4fa1-8dd5-612e43a92461 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:34.173618 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b22d3-1764-4fa1-8dd5-612e43a92461 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:34.173896 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b22d3-1764-4fa1-8dd5-612e43a92461 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:34.232033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b22d3-1764-4fa1-8dd5-612e43a92461 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:34.233466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b22d3-1764-4fa1-8dd5-612e43a92461 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:34.242261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1279/2573] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:34 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:41:38.302051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbd47c-c9d4-4eac-9fe0-fcd4aef4bd51 service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:38.375454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbd47c-c9d4-4eac-9fe0-fcd4aef4bd51 service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:38.382105 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dee5aa-e8be-4fb4-9e36-3d297f1b5e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:38.383861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dee5aa-e8be-4fb4-9e36-3d297f1b5e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:38.384227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dee5aa-e8be-4fb4-9e36-3d297f1b5e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:38.384572 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dee5aa-e8be-4fb4-9e36-3d297f1b5e42 admin admin] PATH_INFO: `/v3/users/7d6f8fdcc3524bafa847b9d8c12e8c12` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:38.386066 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dee5aa-e8be-4fb4-9e36-3d297f1b5e42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6f8fdcc3524bafa847b9d8c12e8c12)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:38.389642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dee5aa-e8be-4fb4-9e36-3d297f1b5e42 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:38.475409 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-94dee5aa-e8be-4fb4-9e36-3d297f1b5e42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6f8fdcc3524bafa847b9d8c12e8c12'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:38.484650 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-94dee5aa-e8be-4fb4-9e36-3d297f1b5e42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6f8fdcc3524bafa847b9d8c12e8c12'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:38.497677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-94dee5aa-e8be-4fb4-9e36-3d297f1b5e42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6f8fdcc3524bafa847b9d8c12e8c12'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:38.499357 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-94dee5aa-e8be-4fb4-9e36-3d297f1b5e42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6f8fdcc3524bafa847b9d8c12e8c12'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:38.499871 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-94dee5aa-e8be-4fb4-9e36-3d297f1b5e42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6f8fdcc3524bafa847b9d8c12e8c12'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:38.524275 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1295/2574] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:41:38 2022] DELETE /identity/v3/users/7d6f8fdcc3524bafa847b9d8c12e8c12 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:38.527895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f9979-c1f8-4857-8530-def8fe3326c2 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:38.528670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f9979-c1f8-4857-8530-def8fe3326c2 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:38.528670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f9979-c1f8-4857-8530-def8fe3326c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:38.548558 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c69f9979-c1f8-4857-8530-def8fe3326c2 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:38.549871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c69f9979-c1f8-4857-8530-def8fe3326c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:38.597182 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:38.661110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1280/2575] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:41:38.672229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dee5aa-e8be-4fb4-9e36-3d297f1b5e42 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:38.677326 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-94dee5aa-e8be-4fb4-9e36-3d297f1b5e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:38.717249 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:38.788685 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dee5aa-e8be-4fb4-9e36-3d297f1b5e42 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:38.796236 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bf984-15e3-44dd-96ce-ed26f144bc1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:38.797810 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bf984-15e3-44dd-96ce-ed26f144bc1c service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:38.798128 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bf984-15e3-44dd-96ce-ed26f144bc1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:38.798334 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bf984-15e3-44dd-96ce-ed26f144bc1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:38.810174 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857bf984-15e3-44dd-96ce-ed26f144bc1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:38.811778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857bf984-15e3-44dd-96ce-ed26f144bc1c service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:38.825841 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1296/2576] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:41:39.147541 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f9979-c1f8-4857-8530-def8fe3326c2 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:39.150264 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c69f9979-c1f8-4857-8530-def8fe3326c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:39.165495 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f9979-c1f8-4857-8530-def8fe3326c2 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:39.171977 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d7b8d-7793-4a42-a96d-549e539204c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:39.173585 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d7b8d-7793-4a42-a96d-549e539204c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:39.174047 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d7b8d-7793-4a42-a96d-549e539204c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:39.174401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d7b8d-7793-4a42-a96d-549e539204c4 admin admin] PATH_INFO: `/v3/projects/90e539bf7c29493a80a26b492c0c3006` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:39.176237 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59d7b8d-7793-4a42-a96d-549e539204c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90e539bf7c29493a80a26b492c0c3006)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:39.179742 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59d7b8d-7793-4a42-a96d-549e539204c4 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:39.254086 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b59d7b8d-7793-4a42-a96d-549e539204c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90e539bf7c29493a80a26b492c0c3006', 'request_id': 'req-b59d7b8d-7793-4a42-a96d-549e539204c4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:39.262383 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b59d7b8d-7793-4a42-a96d-549e539204c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90e539bf7c29493a80a26b492c0c3006', 'request_id': 'req-b59d7b8d-7793-4a42-a96d-549e539204c4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:39.269815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b59d7b8d-7793-4a42-a96d-549e539204c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90e539bf7c29493a80a26b492c0c3006', 'request_id': 'req-b59d7b8d-7793-4a42-a96d-549e539204c4'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:39.285085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b59d7b8d-7793-4a42-a96d-549e539204c4 admin admin] The token cache is being invalidate because project 90e539bf7c29493a80a26b492c0c3006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:41:39.285440 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b59d7b8d-7793-4a42-a96d-549e539204c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:39.286332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-b59d7b8d-7793-4a42-a96d-549e539204c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:39.289530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1281/2577] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:41:39 2022] DELETE /identity/v3/projects/90e539bf7c29493a80a26b492c0c3006 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:39.294977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bf984-15e3-44dd-96ce-ed26f144bc1c service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:39.312301 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bf984-15e3-44dd-96ce-ed26f144bc1c service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:39.400392 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02010bc6-f458-42ab-a288-88e584094c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:39.401844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02010bc6-f458-42ab-a288-88e584094c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:39.402362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02010bc6-f458-42ab-a288-88e584094c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:39.402835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02010bc6-f458-42ab-a288-88e584094c9c admin admin] PATH_INFO: `/v3/users/282ff58f93774ddaac047a729fdb0977` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:39.404630 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02010bc6-f458-42ab-a288-88e584094c9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=282ff58f93774ddaac047a729fdb0977)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:39.407112 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02010bc6-f458-42ab-a288-88e584094c9c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:39.477095 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-02010bc6-f458-42ab-a288-88e584094c9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '282ff58f93774ddaac047a729fdb0977'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:39.496134 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-02010bc6-f458-42ab-a288-88e584094c9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '282ff58f93774ddaac047a729fdb0977'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:39.504512 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-02010bc6-f458-42ab-a288-88e584094c9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '282ff58f93774ddaac047a729fdb0977'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:39.505177 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-02010bc6-f458-42ab-a288-88e584094c9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '282ff58f93774ddaac047a729fdb0977'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:39.506002 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-02010bc6-f458-42ab-a288-88e584094c9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '282ff58f93774ddaac047a729fdb0977'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:39.525949 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1297/2578] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:41:39 2022] DELETE /identity/v3/users/282ff58f93774ddaac047a729fdb0977 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:39.855618 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d7b8d-7793-4a42-a96d-549e539204c4 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:39.857227 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b59d7b8d-7793-4a42-a96d-549e539204c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:39.872181 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d7b8d-7793-4a42-a96d-549e539204c4 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:39.935961 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12602112-53da-44d2-b364-e4e18068b2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:39.936839 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12602112-53da-44d2-b364-e4e18068b2fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:39.937127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12602112-53da-44d2-b364-e4e18068b2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:39.937543 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12602112-53da-44d2-b364-e4e18068b2fe admin admin] PATH_INFO: `/v3/projects/1c65bd03ee674156bf390dacaf21f43b` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:39.938783 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12602112-53da-44d2-b364-e4e18068b2fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c65bd03ee674156bf390dacaf21f43b)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:39.940763 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12602112-53da-44d2-b364-e4e18068b2fe admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:39.995475 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-12602112-53da-44d2-b364-e4e18068b2fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c65bd03ee674156bf390dacaf21f43b', 'request_id': 'req-12602112-53da-44d2-b364-e4e18068b2fe'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:40.007647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-12602112-53da-44d2-b364-e4e18068b2fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c65bd03ee674156bf390dacaf21f43b', 'request_id': 'req-12602112-53da-44d2-b364-e4e18068b2fe'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:40.018599 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-12602112-53da-44d2-b364-e4e18068b2fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c65bd03ee674156bf390dacaf21f43b', 'request_id': 'req-12602112-53da-44d2-b364-e4e18068b2fe'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:40.031121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:40.039490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-12602112-53da-44d2-b364-e4e18068b2fe admin admin] The token cache is being invalidate because project 1c65bd03ee674156bf390dacaf21f43b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:41:40.040104 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-12602112-53da-44d2-b364-e4e18068b2fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:40.041041 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-12602112-53da-44d2-b364-e4e18068b2fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:40.042682 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1282/2579] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:41:39 2022] DELETE /identity/v3/projects/1c65bd03ee674156bf390dacaf21f43b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:40.047720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9afe1e-9ca7-41db-8e5b-16d80900abcf None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:40.048007 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9afe1e-9ca7-41db-8e5b-16d80900abcf None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:40.048310 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9afe1e-9ca7-41db-8e5b-16d80900abcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:40.066138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-1d9afe1e-9ca7-41db-8e5b-16d80900abcf None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:40.067382 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-1d9afe1e-9ca7-41db-8e5b-16d80900abcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:40.101454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:40.140498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1298/2580] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:40.150287 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841def2-7012-4fa1-a6d0-1853412e37ab None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:40.150582 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841def2-7012-4fa1-a6d0-1853412e37ab None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:40.150859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841def2-7012-4fa1-a6d0-1853412e37ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:40.170653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e841def2-7012-4fa1-a6d0-1853412e37ab None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:40.172030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e841def2-7012-4fa1-a6d0-1853412e37ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:40.180174 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1283/2581] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:40.184623 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9afe1e-9ca7-41db-8e5b-16d80900abcf None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:40.199062 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9afe1e-9ca7-41db-8e5b-16d80900abcf None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:40.204756 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209de05-20de-4956-8688-261811553dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:40.205614 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209de05-20de-4956-8688-261811553dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:40.205991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209de05-20de-4956-8688-261811553dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:40.206522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209de05-20de-4956-8688-261811553dd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:40.207545 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209de05-20de-4956-8688-261811553dd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:41:40.207924 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209de05-20de-4956-8688-261811553dd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:40.209990 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209de05-20de-4956-8688-261811553dd2 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:40.235674 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1299/2582] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:41:40.240438 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841def2-7012-4fa1-a6d0-1853412e37ab None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:40.251452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841def2-7012-4fa1-a6d0-1853412e37ab None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:40.257348 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a8a11-fc10-49df-b33e-095580c787ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:40.258091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a8a11-fc10-49df-b33e-095580c787ad admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:40.258372 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a8a11-fc10-49df-b33e-095580c787ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:40.258651 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a8a11-fc10-49df-b33e-095580c787ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:40.259428 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a8a11-fc10-49df-b33e-095580c787ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:40.261981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a8a11-fc10-49df-b33e-095580c787ad admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:40.294260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1284/2583] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:41:40 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:41:40.298300 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209de05-20de-4956-8688-261811553dd2 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:40.310442 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209de05-20de-4956-8688-261811553dd2 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:40.386635 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a9327-74ee-411e-bdde-05d3e0d04217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:40.387346 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a9327-74ee-411e-bdde-05d3e0d04217 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:40.387628 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a9327-74ee-411e-bdde-05d3e0d04217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:40.387906 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a9327-74ee-411e-bdde-05d3e0d04217 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:40.388609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a9327-74ee-411e-bdde-05d3e0d04217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:40.390573 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a9327-74ee-411e-bdde-05d3e0d04217 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:40.411143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1300/2584] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:41:40 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:41:40.415295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a8a11-fc10-49df-b33e-095580c787ad admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:40.425849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a8a11-fc10-49df-b33e-095580c787ad admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:40.431409 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb27a82-135c-484f-a0f0-bba910dfcd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:40.432074 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb27a82-135c-484f-a0f0-bba910dfcd64 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:40.432351 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb27a82-135c-484f-a0f0-bba910dfcd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:40.432638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb27a82-135c-484f-a0f0-bba910dfcd64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:40.433903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb27a82-135c-484f-a0f0-bba910dfcd64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:40.435545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb27a82-135c-484f-a0f0-bba910dfcd64 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:40.446663 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1285/2585] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:40.451477 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a9327-74ee-411e-bdde-05d3e0d04217 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:40.462096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a9327-74ee-411e-bdde-05d3e0d04217 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:40.467035 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e674cdf-c3a8-4638-8fea-11e15ec41aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:40.468603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e674cdf-c3a8-4638-8fea-11e15ec41aed admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:40.469070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e674cdf-c3a8-4638-8fea-11e15ec41aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:40.469425 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e674cdf-c3a8-4638-8fea-11e15ec41aed admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:40.470178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e674cdf-c3a8-4638-8fea-11e15ec41aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:40.471859 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e674cdf-c3a8-4638-8fea-11e15ec41aed admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:40.484006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1301/2586] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:41:40.488212 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb27a82-135c-484f-a0f0-bba910dfcd64 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:40.499226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb27a82-135c-484f-a0f0-bba910dfcd64 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:40.504103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae66926-df83-49c3-adec-0cdbc8b3758c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:40.504975 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae66926-df83-49c3-adec-0cdbc8b3758c admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:40.505255 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae66926-df83-49c3-adec-0cdbc8b3758c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:40.505532 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae66926-df83-49c3-adec-0cdbc8b3758c admin admin] PATH_INFO: `/v3/projects/d9a058dbe5274e73b8743b064fb887be/users/1190fbd6a1874b84bc504f4e864368be/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:40.521598 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae66926-df83-49c3-adec-0cdbc8b3758c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9a058dbe5274e73b8743b064fb887be, user_id=1190fbd6a1874b84bc504f4e864368be, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:40.524240 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae66926-df83-49c3-adec-0cdbc8b3758c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:40.537410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1286/2587] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:41:40 2022] PUT /identity/v3/projects/d9a058dbe5274e73b8743b064fb887be/users/1190fbd6a1874b84bc504f4e864368be/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:40.547947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b23168-44ed-49d0-9b95-de121db287cb None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:40.548276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b23168-44ed-49d0-9b95-de121db287cb None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:40.548595 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b23168-44ed-49d0-9b95-de121db287cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:40.578485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-41b23168-44ed-49d0-9b95-de121db287cb None None] MFA Rules not processed for user `1190fbd6a1874b84bc504f4e864368be`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:40.579748 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-41b23168-44ed-49d0-9b95-de121db287cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:40.651350 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1302/2588] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:40.655827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae66926-df83-49c3-adec-0cdbc8b3758c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:40.732687 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae66926-df83-49c3-adec-0cdbc8b3758c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:40.737675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c175ebe-8c4c-4891-adab-61ef78a78a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:40.738343 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c175ebe-8c4c-4891-adab-61ef78a78a8e admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:40.738625 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c175ebe-8c4c-4891-adab-61ef78a78a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:40.738901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c175ebe-8c4c-4891-adab-61ef78a78a8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:40.739632 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c175ebe-8c4c-4891-adab-61ef78a78a8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:40.741569 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c175ebe-8c4c-4891-adab-61ef78a78a8e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:40.768644 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1287/2589] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:41:40 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:41:40.773048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b23168-44ed-49d0-9b95-de121db287cb None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:40.785126 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b23168-44ed-49d0-9b95-de121db287cb None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:40.818204 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:40.877345 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6a2b6-17d0-4f9f-81ae-2c7efcd2095c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:40.878182 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6a2b6-17d0-4f9f-81ae-2c7efcd2095c admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:40.878474 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6a2b6-17d0-4f9f-81ae-2c7efcd2095c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:40.878757 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6a2b6-17d0-4f9f-81ae-2c7efcd2095c admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:40.879666 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa6a2b6-17d0-4f9f-81ae-2c7efcd2095c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:40.882343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa6a2b6-17d0-4f9f-81ae-2c7efcd2095c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:40.914866 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1303/2590] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:41:40 2022] POST /identity/v3/users => generated 298 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:41:40.920522 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c175ebe-8c4c-4891-adab-61ef78a78a8e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:40.932811 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c175ebe-8c4c-4891-adab-61ef78a78a8e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:40.942590 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409066d-5816-457f-a742-b2bd81c8cf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:40.943305 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409066d-5816-457f-a742-b2bd81c8cf0b admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:40.945086 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409066d-5816-457f-a742-b2bd81c8cf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:40.945086 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409066d-5816-457f-a742-b2bd81c8cf0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:40.945086 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9409066d-5816-457f-a742-b2bd81c8cf0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:40.946386 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9409066d-5816-457f-a742-b2bd81c8cf0b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:40.957708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1288/2591] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:40.962025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6a2b6-17d0-4f9f-81ae-2c7efcd2095c admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:40.973854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6a2b6-17d0-4f9f-81ae-2c7efcd2095c admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:40.979023 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb0fc7-9485-46c6-b367-a0a252baaaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:40.980896 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb0fc7-9485-46c6-b367-a0a252baaaec admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:40.981328 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb0fc7-9485-46c6-b367-a0a252baaaec admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:40.981681 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb0fc7-9485-46c6-b367-a0a252baaaec admin admin] PATH_INFO: `/v3/projects/abaf5e9490fb4872babfd97e2dcc358c/users/cfc96324fba943bdbdcd3e20b919fccf/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:41.001117 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb0fc7-9485-46c6-b367-a0a252baaaec admin admin] RBAC: Authorizing `identity:create_grant(project_id=abaf5e9490fb4872babfd97e2dcc358c, user_id=cfc96324fba943bdbdcd3e20b919fccf, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:41.007817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb0fc7-9485-46c6-b367-a0a252baaaec admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:41.029544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1304/2592] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:41:40 2022] PUT /identity/v3/projects/abaf5e9490fb4872babfd97e2dcc358c/users/cfc96324fba943bdbdcd3e20b919fccf/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:41.030862 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409066d-5816-457f-a742-b2bd81c8cf0b admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:41.043847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409066d-5816-457f-a742-b2bd81c8cf0b admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:41.145112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3209850-9a0d-4055-bfe7-e515bcf45c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:41.145789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3209850-9a0d-4055-bfe7-e515bcf45c30 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:41.146067 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3209850-9a0d-4055-bfe7-e515bcf45c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:41.146339 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3209850-9a0d-4055-bfe7-e515bcf45c30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:41.147238 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3209850-9a0d-4055-bfe7-e515bcf45c30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:41.148823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3209850-9a0d-4055-bfe7-e515bcf45c30 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:41.159258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1289/2593] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:41.163518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb0fc7-9485-46c6-b367-a0a252baaaec admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:41.174576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb0fc7-9485-46c6-b367-a0a252baaaec admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:41.179213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb05ad-7e38-41bb-8fd9-33b4eb803ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:41.180088 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb05ad-7e38-41bb-8fd9-33b4eb803ad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:41.180375 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb05ad-7e38-41bb-8fd9-33b4eb803ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:41.180651 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb05ad-7e38-41bb-8fd9-33b4eb803ad3 admin admin] PATH_INFO: `/v3/projects/abaf5e9490fb4872babfd97e2dcc358c/users/cfc96324fba943bdbdcd3e20b919fccf/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:41.182649 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb05ad-7e38-41bb-8fd9-33b4eb803ad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abaf5e9490fb4872babfd97e2dcc358c, user_id=cfc96324fba943bdbdcd3e20b919fccf, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:41.185320 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb05ad-7e38-41bb-8fd9-33b4eb803ad3 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:41.195454 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1305/2594] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:41:41 2022] PUT /identity/v3/projects/abaf5e9490fb4872babfd97e2dcc358c/users/cfc96324fba943bdbdcd3e20b919fccf/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:41.206877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a0c72-119b-4212-9f0e-f95f36bb90f1 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:41.207206 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a0c72-119b-4212-9f0e-f95f36bb90f1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:41.207626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a0c72-119b-4212-9f0e-f95f36bb90f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:41.215154 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:41.236405 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a69a0c72-119b-4212-9f0e-f95f36bb90f1 None None] MFA Rules not processed for user `cfc96324fba943bdbdcd3e20b919fccf`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:41.237650 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a69a0c72-119b-4212-9f0e-f95f36bb90f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:41.305124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1290/2595] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:41.317984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb05ad-7e38-41bb-8fd9-33b4eb803ad3 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:41.319829 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-febb05ad-7e38-41bb-8fd9-33b4eb803ad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:41.381464 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb05ad-7e38-41bb-8fd9-33b4eb803ad3 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:41.385474 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b6db3-e3f3-4c7f-bdbd-fb6bad894faf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:41.386152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b6db3-e3f3-4c7f-bdbd-fb6bad894faf service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:41.386440 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b6db3-e3f3-4c7f-bdbd-fb6bad894faf service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:41.386717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b6db3-e3f3-4c7f-bdbd-fb6bad894faf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:41.449927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b6db3-e3f3-4c7f-bdbd-fb6bad894faf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:41.451417 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b6db3-e3f3-4c7f-bdbd-fb6bad894faf service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:41.461208 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1306/2596] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:41:45.888292 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a0c72-119b-4212-9f0e-f95f36bb90f1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:45.903168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a0c72-119b-4212-9f0e-f95f36bb90f1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:45.910809 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc22676-33e8-4816-a1cc-de86f75c10fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:45.912457 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc22676-33e8-4816-a1cc-de86f75c10fa service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:45.913076 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc22676-33e8-4816-a1cc-de86f75c10fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:45.913950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc22676-33e8-4816-a1cc-de86f75c10fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:45.939945 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc22676-33e8-4816-a1cc-de86f75c10fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:45.941745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc22676-33e8-4816-a1cc-de86f75c10fa service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:45.965467 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1291/2597] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:41:46.059676 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3563cf-ef61-487c-b4aa-2686125b0787 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:46.060772 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3563cf-ef61-487c-b4aa-2686125b0787 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:46.061247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3563cf-ef61-487c-b4aa-2686125b0787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:46.082041 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-ba3563cf-ef61-487c-b4aa-2686125b0787 None None] MFA Rules not processed for user `4f75b8b484384554ad59aa20535875e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:46.083787 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ba3563cf-ef61-487c-b4aa-2686125b0787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:46.136359 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:46.183123 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1307/2598] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jul 14 20:41:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:41:46.212307 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc22676-33e8-4816-a1cc-de86f75c10fa service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:46.214456 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8bc22676-33e8-4816-a1cc-de86f75c10fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:46.426429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc22676-33e8-4816-a1cc-de86f75c10fa service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:46.444249 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a23116-d571-4836-b04c-5136c6fd227a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc23233929144a5b7b3a1733da19828', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:46.445859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a23116-d571-4836-b04c-5136c6fd227a service swift] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:46.447164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a23116-d571-4836-b04c-5136c6fd227a service swift] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:46.448470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a23116-d571-4836-b04c-5136c6fd227a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:46.460931 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a23116-d571-4836-b04c-5136c6fd227a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:46.463295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a23116-d571-4836-b04c-5136c6fd227a service swift] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:46.490825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1292/2599] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 14 20:41:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 20:41:50.626242 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3563cf-ef61-487c-b4aa-2686125b0787 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:50.627911 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ba3563cf-ef61-487c-b4aa-2686125b0787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:50.708365 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3563cf-ef61-487c-b4aa-2686125b0787 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:50.713224 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e84d6-0a0f-4d7e-ba51-bf7e49863fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:50.714420 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e84d6-0a0f-4d7e-ba51-bf7e49863fdf service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:50.714701 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e84d6-0a0f-4d7e-ba51-bf7e49863fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:50.715010 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e84d6-0a0f-4d7e-ba51-bf7e49863fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:50.716858 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-a32e84d6-0a0f-4d7e-ba51-bf7e49863fdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:50.789537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e84d6-0a0f-4d7e-ba51-bf7e49863fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:50.791215 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e84d6-0a0f-4d7e-ba51-bf7e49863fdf service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:50.802877 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1308/2600] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:41:51.271807 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a23116-d571-4836-b04c-5136c6fd227a service swift] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:51.285288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a23116-d571-4836-b04c-5136c6fd227a service swift] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:51.292121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327e645-ff2e-4378-9343-9d73c898c361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:51.293223 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327e645-ff2e-4378-9343-9d73c898c361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:51.293518 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327e645-ff2e-4378-9343-9d73c898c361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:51.293827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327e645-ff2e-4378-9343-9d73c898c361 admin admin] PATH_INFO: `/v3/users/1190fbd6a1874b84bc504f4e864368be` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:51.295116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327e645-ff2e-4378-9343-9d73c898c361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1190fbd6a1874b84bc504f4e864368be)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:51.297315 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327e645-ff2e-4378-9343-9d73c898c361 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:51.353443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8327e645-ff2e-4378-9343-9d73c898c361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1190fbd6a1874b84bc504f4e864368be'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:51.364147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8327e645-ff2e-4378-9343-9d73c898c361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1190fbd6a1874b84bc504f4e864368be'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:51.365018 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8327e645-ff2e-4378-9343-9d73c898c361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1190fbd6a1874b84bc504f4e864368be'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:51.365943 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8327e645-ff2e-4378-9343-9d73c898c361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1190fbd6a1874b84bc504f4e864368be'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:51.372897 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:51.389329 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8327e645-ff2e-4378-9343-9d73c898c361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1190fbd6a1874b84bc504f4e864368be'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:51.406985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1293/2601] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:41:51 2022] DELETE /identity/v3/users/1190fbd6a1874b84bc504f4e864368be => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:51.411094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d7e83-3fcb-4dd0-8160-bb92f3d09f52 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:51.411389 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d7e83-3fcb-4dd0-8160-bb92f3d09f52 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:51.411666 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d7e83-3fcb-4dd0-8160-bb92f3d09f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:51.433144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-072d7e83-3fcb-4dd0-8160-bb92f3d09f52 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:51.434530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-072d7e83-3fcb-4dd0-8160-bb92f3d09f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:51.508067 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1309/2602] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:51.520755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327e645-ff2e-4378-9343-9d73c898c361 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:51.522234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8327e645-ff2e-4378-9343-9d73c898c361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:51.593461 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327e645-ff2e-4378-9343-9d73c898c361 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:51.597524 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd7809-8a53-42ac-acf4-d0cadc8933f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:51.598220 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd7809-8a53-42ac-acf4-d0cadc8933f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:51.598503 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd7809-8a53-42ac-acf4-d0cadc8933f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:51.598773 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd7809-8a53-42ac-acf4-d0cadc8933f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:51.608954 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd7809-8a53-42ac-acf4-d0cadc8933f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:51.610421 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd7809-8a53-42ac-acf4-d0cadc8933f8 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:51.622345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1294/2603] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:41:52.247435 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d7e83-3fcb-4dd0-8160-bb92f3d09f52 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:52.249502 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-072d7e83-3fcb-4dd0-8160-bb92f3d09f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:52.262219 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d7e83-3fcb-4dd0-8160-bb92f3d09f52 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:52.281139 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc288b-7fd8-4309-b20e-27d2fd43bcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:52.282584 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc288b-7fd8-4309-b20e-27d2fd43bcb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:52.284292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc288b-7fd8-4309-b20e-27d2fd43bcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:52.284580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc288b-7fd8-4309-b20e-27d2fd43bcb7 admin admin] PATH_INFO: `/v3/projects/d9a058dbe5274e73b8743b064fb887be` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:52.286034 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc288b-7fd8-4309-b20e-27d2fd43bcb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9a058dbe5274e73b8743b064fb887be)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:52.288066 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc288b-7fd8-4309-b20e-27d2fd43bcb7 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:52.324098 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:52.352212 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ecfc288b-7fd8-4309-b20e-27d2fd43bcb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9a058dbe5274e73b8743b064fb887be', 'request_id': 'req-ecfc288b-7fd8-4309-b20e-27d2fd43bcb7'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:52.363866 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ecfc288b-7fd8-4309-b20e-27d2fd43bcb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9a058dbe5274e73b8743b064fb887be', 'request_id': 'req-ecfc288b-7fd8-4309-b20e-27d2fd43bcb7'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:52.371736 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ecfc288b-7fd8-4309-b20e-27d2fd43bcb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9a058dbe5274e73b8743b064fb887be', 'request_id': 'req-ecfc288b-7fd8-4309-b20e-27d2fd43bcb7'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:52.383830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ecfc288b-7fd8-4309-b20e-27d2fd43bcb7 admin admin] The token cache is being invalidate because project d9a058dbe5274e73b8743b064fb887be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:41:52.384221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ecfc288b-7fd8-4309-b20e-27d2fd43bcb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:52.384990 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-ecfc288b-7fd8-4309-b20e-27d2fd43bcb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:52.387771 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1310/2604] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:41:52 2022] DELETE /identity/v3/projects/d9a058dbe5274e73b8743b064fb887be => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:52.392230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd7809-8a53-42ac-acf4-d0cadc8933f8 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:52.404147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd7809-8a53-42ac-acf4-d0cadc8933f8 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:52.563517 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c5a8c-fd32-480a-b775-ae6ed08f55a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:52.563955 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c5a8c-fd32-480a-b775-ae6ed08f55a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:52.564243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c5a8c-fd32-480a-b775-ae6ed08f55a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:52.564519 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c5a8c-fd32-480a-b775-ae6ed08f55a2 admin admin] PATH_INFO: `/v3/users/cfc96324fba943bdbdcd3e20b919fccf` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:52.566150 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c5a8c-fd32-480a-b775-ae6ed08f55a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfc96324fba943bdbdcd3e20b919fccf)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:52.569415 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c5a8c-fd32-480a-b775-ae6ed08f55a2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:52.659153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8d3c5a8c-fd32-480a-b775-ae6ed08f55a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfc96324fba943bdbdcd3e20b919fccf'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:52.671203 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8d3c5a8c-fd32-480a-b775-ae6ed08f55a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfc96324fba943bdbdcd3e20b919fccf'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:52.672429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8d3c5a8c-fd32-480a-b775-ae6ed08f55a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfc96324fba943bdbdcd3e20b919fccf'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:52.673195 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8d3c5a8c-fd32-480a-b775-ae6ed08f55a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfc96324fba943bdbdcd3e20b919fccf'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:52.708834 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-8d3c5a8c-fd32-480a-b775-ae6ed08f55a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfc96324fba943bdbdcd3e20b919fccf'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:52.724097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:52.736434 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1295/2605] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:41:52 2022] DELETE /identity/v3/users/cfc96324fba943bdbdcd3e20b919fccf => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:53.464525 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc288b-7fd8-4309-b20e-27d2fd43bcb7 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:53.469724 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ecfc288b-7fd8-4309-b20e-27d2fd43bcb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:53.508376 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc288b-7fd8-4309-b20e-27d2fd43bcb7 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:53.588108 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d798c43-5806-43bd-b565-c3964ef98747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:53.589655 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d798c43-5806-43bd-b565-c3964ef98747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:53.590010 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d798c43-5806-43bd-b565-c3964ef98747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:53.590498 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d798c43-5806-43bd-b565-c3964ef98747 admin admin] PATH_INFO: `/v3/projects/abaf5e9490fb4872babfd97e2dcc358c` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:53.592086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d798c43-5806-43bd-b565-c3964ef98747 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abaf5e9490fb4872babfd97e2dcc358c)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:53.594294 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d798c43-5806-43bd-b565-c3964ef98747 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:53.671449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8d798c43-5806-43bd-b565-c3964ef98747 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abaf5e9490fb4872babfd97e2dcc358c', 'request_id': 'req-8d798c43-5806-43bd-b565-c3964ef98747'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:53.680703 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8d798c43-5806-43bd-b565-c3964ef98747 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abaf5e9490fb4872babfd97e2dcc358c', 'request_id': 'req-8d798c43-5806-43bd-b565-c3964ef98747'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:53.699573 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8d798c43-5806-43bd-b565-c3964ef98747 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abaf5e9490fb4872babfd97e2dcc358c', 'request_id': 'req-8d798c43-5806-43bd-b565-c3964ef98747'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:53.717241 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8d798c43-5806-43bd-b565-c3964ef98747 admin admin] The token cache is being invalidate because project abaf5e9490fb4872babfd97e2dcc358c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:41:53.717606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8d798c43-5806-43bd-b565-c3964ef98747 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:53.718218 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8d798c43-5806-43bd-b565-c3964ef98747 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:53.720148 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1311/2606] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:41:53 2022] DELETE /identity/v3/projects/abaf5e9490fb4872babfd97e2dcc358c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:53.725939 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a36637-275a-4890-8ab3-b440b174498a None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:53.726425 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a36637-275a-4890-8ab3-b440b174498a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:53.726508 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a36637-275a-4890-8ab3-b440b174498a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:53.756067 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-53a36637-275a-4890-8ab3-b440b174498a None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:53.757992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-53a36637-275a-4890-8ab3-b440b174498a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:53.841619 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1296/2607] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:53.853465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3cad8-6b8c-4e6f-b952-f44d234a4b0b None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:53.853465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3cad8-6b8c-4e6f-b952-f44d234a4b0b None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:53.853465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3cad8-6b8c-4e6f-b952-f44d234a4b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:53.881610 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-69d3cad8-6b8c-4e6f-b952-f44d234a4b0b None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:53.899965 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-69d3cad8-6b8c-4e6f-b952-f44d234a4b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:53.919263 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1312/2608] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:53.923403 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a36637-275a-4890-8ab3-b440b174498a None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:53.935491 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a36637-275a-4890-8ab3-b440b174498a None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:53.943139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012fae1-fd8b-4aa1-a75c-527c2ad9638d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:53.944086 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012fae1-fd8b-4aa1-a75c-527c2ad9638d admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:53.944716 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012fae1-fd8b-4aa1-a75c-527c2ad9638d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:53.945073 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012fae1-fd8b-4aa1-a75c-527c2ad9638d admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:53.946121 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012fae1-fd8b-4aa1-a75c-527c2ad9638d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:41:53.946515 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012fae1-fd8b-4aa1-a75c-527c2ad9638d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:53.948512 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012fae1-fd8b-4aa1-a75c-527c2ad9638d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:53.965589 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1297/2609] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:41:53.972021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3cad8-6b8c-4e6f-b952-f44d234a4b0b None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:53.988229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3cad8-6b8c-4e6f-b952-f44d234a4b0b None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:53.998072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79cfa7-0e62-4882-a2ea-d2096151ba80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:53.999340 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79cfa7-0e62-4882-a2ea-d2096151ba80 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:53.999523 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79cfa7-0e62-4882-a2ea-d2096151ba80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:53.999804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79cfa7-0e62-4882-a2ea-d2096151ba80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:54.000989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79cfa7-0e62-4882-a2ea-d2096151ba80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:54.004762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79cfa7-0e62-4882-a2ea-d2096151ba80 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:54.087326 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1313/2610] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:41:53 2022] POST /identity/v3/projects => generated 361 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:41:54.093612 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012fae1-fd8b-4aa1-a75c-527c2ad9638d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:54.112551 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012fae1-fd8b-4aa1-a75c-527c2ad9638d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:54.251002 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01bbf1-1215-4168-86c6-845854c27e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:54.252307 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01bbf1-1215-4168-86c6-845854c27e23 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:54.253291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01bbf1-1215-4168-86c6-845854c27e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:54.253767 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01bbf1-1215-4168-86c6-845854c27e23 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:54.254686 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01bbf1-1215-4168-86c6-845854c27e23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:54.256969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01bbf1-1215-4168-86c6-845854c27e23 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:54.293016 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1298/2611] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:41:54 2022] POST /identity/v3/users => generated 281 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:41:54.296884 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79cfa7-0e62-4882-a2ea-d2096151ba80 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:54.313463 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79cfa7-0e62-4882-a2ea-d2096151ba80 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:54.319929 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721069f-8ac4-483b-a8b2-5ca00383d356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:54.320737 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721069f-8ac4-483b-a8b2-5ca00383d356 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:54.321101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721069f-8ac4-483b-a8b2-5ca00383d356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:54.321457 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721069f-8ac4-483b-a8b2-5ca00383d356 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:54.322330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721069f-8ac4-483b-a8b2-5ca00383d356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:54.324135 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721069f-8ac4-483b-a8b2-5ca00383d356 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:54.336596 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1314/2612] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:54.341365 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01bbf1-1215-4168-86c6-845854c27e23 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:54.355059 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01bbf1-1215-4168-86c6-845854c27e23 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:54.376185 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ed6ad-887e-47f8-ad15-726370d45417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:54.381632 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ed6ad-887e-47f8-ad15-726370d45417 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:54.381918 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ed6ad-887e-47f8-ad15-726370d45417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:54.382203 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ed6ad-887e-47f8-ad15-726370d45417 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:54.383025 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ed6ad-887e-47f8-ad15-726370d45417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:54.384737 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ed6ad-887e-47f8-ad15-726370d45417 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:54.418041 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1299/2613] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:54 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:54.419508 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721069f-8ac4-483b-a8b2-5ca00383d356 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:54.427016 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:54.434974 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721069f-8ac4-483b-a8b2-5ca00383d356 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:54.457778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bec55-82b5-4c12-ad16-0aded2f3c6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:54.458726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bec55-82b5-4c12-ad16-0aded2f3c6a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:54.459382 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bec55-82b5-4c12-ad16-0aded2f3c6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:54.459907 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bec55-82b5-4c12-ad16-0aded2f3c6a9 admin admin] PATH_INFO: `/v3/projects/db8108a8afa9403aa78475c06496b951/users/d9fa00514155444489338ff20ecfd862/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:54.495861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bec55-82b5-4c12-ad16-0aded2f3c6a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db8108a8afa9403aa78475c06496b951, user_id=d9fa00514155444489338ff20ecfd862, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:54.502287 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bec55-82b5-4c12-ad16-0aded2f3c6a9 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:54.520214 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1315/2614] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:41:54 2022] PUT /identity/v3/projects/db8108a8afa9403aa78475c06496b951/users/d9fa00514155444489338ff20ecfd862/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:54.599199 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d859838-e3df-43b7-b0ac-d53b6f7499ad None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:54.599535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d859838-e3df-43b7-b0ac-d53b6f7499ad None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:54.599758 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d859838-e3df-43b7-b0ac-d53b6f7499ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:54.661067 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-7d859838-e3df-43b7-b0ac-d53b6f7499ad None None] MFA Rules not processed for user `d9fa00514155444489338ff20ecfd862`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:54.662381 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7d859838-e3df-43b7-b0ac-d53b6f7499ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:54.755581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:54.778495 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1300/2615] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:54.819337 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bec55-82b5-4c12-ad16-0aded2f3c6a9 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:54.821352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3d1bec55-82b5-4c12-ad16-0aded2f3c6a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:54.936145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d859838-e3df-43b7-b0ac-d53b6f7499ad None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:54.938210 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7d859838-e3df-43b7-b0ac-d53b6f7499ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:55.004665 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bec55-82b5-4c12-ad16-0aded2f3c6a9 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:55.015604 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839e8ce-fb0d-4897-ad6b-e35b3396fa63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:55.016284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839e8ce-fb0d-4897-ad6b-e35b3396fa63 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:55.016748 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839e8ce-fb0d-4897-ad6b-e35b3396fa63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:55.017043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839e8ce-fb0d-4897-ad6b-e35b3396fa63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:55.033137 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839e8ce-fb0d-4897-ad6b-e35b3396fa63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:55.033137 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839e8ce-fb0d-4897-ad6b-e35b3396fa63 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:55.050051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1316/2616] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:41:55.050927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839e8ce-fb0d-4897-ad6b-e35b3396fa63 service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:55.052678 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f839e8ce-fb0d-4897-ad6b-e35b3396fa63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:55.099451 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d859838-e3df-43b7-b0ac-d53b6f7499ad None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:55.108821 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427c40e-2f9e-4fc2-a625-2c50cd6ee47a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afad68819504c2b864851c6989ae25a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:55.108821 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427c40e-2f9e-4fc2-a625-2c50cd6ee47a service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:55.109364 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427c40e-2f9e-4fc2-a625-2c50cd6ee47a service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:55.109364 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427c40e-2f9e-4fc2-a625-2c50cd6ee47a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:55.112496 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4427c40e-2f9e-4fc2-a625-2c50cd6ee47a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:55.126779 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839e8ce-fb0d-4897-ad6b-e35b3396fa63 service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:55.127108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427c40e-2f9e-4fc2-a625-2c50cd6ee47a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:55.128731 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427c40e-2f9e-4fc2-a625-2c50cd6ee47a service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:55.131805 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b400f-567d-4624-a985-fbcfa8060c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:55.132681 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b400f-567d-4624-a985-fbcfa8060c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:55.132965 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b400f-567d-4624-a985-fbcfa8060c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:55.133243 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b400f-567d-4624-a985-fbcfa8060c86 admin admin] PATH_INFO: `/v3/users/5cf43634335a428093ee30c2dd99187b` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:55.135213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b400f-567d-4624-a985-fbcfa8060c86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cf43634335a428093ee30c2dd99187b)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:55.136902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b400f-567d-4624-a985-fbcfa8060c86 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:55.140447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1301/2617] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:41:55.141313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427c40e-2f9e-4fc2-a625-2c50cd6ee47a service placement] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:55.142678 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4427c40e-2f9e-4fc2-a625-2c50cd6ee47a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:55.162516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427c40e-2f9e-4fc2-a625-2c50cd6ee47a service placement] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:55.167920 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c53fcd-84ce-4a83-86b6-58389251a3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:55.168801 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c53fcd-84ce-4a83-86b6-58389251a3a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:55.169090 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c53fcd-84ce-4a83-86b6-58389251a3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:55.169362 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c53fcd-84ce-4a83-86b6-58389251a3a2 admin admin] PATH_INFO: `/v3/users/75744cbe26cb4635af3db596f3c235b5` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:55.170706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c53fcd-84ce-4a83-86b6-58389251a3a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75744cbe26cb4635af3db596f3c235b5)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:55.172695 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c53fcd-84ce-4a83-86b6-58389251a3a2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:55.208577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-9a8b400f-567d-4624-a985-fbcfa8060c86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cf43634335a428093ee30c2dd99187b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.224290 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-9a8b400f-567d-4624-a985-fbcfa8060c86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cf43634335a428093ee30c2dd99187b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.236098 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-9a8b400f-567d-4624-a985-fbcfa8060c86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cf43634335a428093ee30c2dd99187b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.237153 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-9a8b400f-567d-4624-a985-fbcfa8060c86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cf43634335a428093ee30c2dd99187b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.237961 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-9a8b400f-567d-4624-a985-fbcfa8060c86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cf43634335a428093ee30c2dd99187b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.238567 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d8c53fcd-84ce-4a83-86b6-58389251a3a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75744cbe26cb4635af3db596f3c235b5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.247859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d8c53fcd-84ce-4a83-86b6-58389251a3a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75744cbe26cb4635af3db596f3c235b5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.248505 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d8c53fcd-84ce-4a83-86b6-58389251a3a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75744cbe26cb4635af3db596f3c235b5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.249143 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d8c53fcd-84ce-4a83-86b6-58389251a3a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75744cbe26cb4635af3db596f3c235b5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.262993 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-d8c53fcd-84ce-4a83-86b6-58389251a3a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75744cbe26cb4635af3db596f3c235b5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.263358 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1317/2618] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:41:55 2022] DELETE /identity/v3/users/5cf43634335a428093ee30c2dd99187b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:55.264830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b400f-567d-4624-a985-fbcfa8060c86 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:55.267132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9a8b400f-567d-4624-a985-fbcfa8060c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:55.274879 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:55.286566 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1302/2619] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:41:55 2022] DELETE /identity/v3/users/75744cbe26cb4635af3db596f3c235b5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:55.362888 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b400f-567d-4624-a985-fbcfa8060c86 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:55.426367 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bbe68-1fe0-4790-9f5e-467f15e853a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:55.427399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bbe68-1fe0-4790-9f5e-467f15e853a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:55.427686 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bbe68-1fe0-4790-9f5e-467f15e853a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:55.427966 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bbe68-1fe0-4790-9f5e-467f15e853a4 admin admin] PATH_INFO: `/v3/users/d9fa00514155444489338ff20ecfd862` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:55.429807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bbe68-1fe0-4790-9f5e-467f15e853a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9fa00514155444489338ff20ecfd862)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:55.431921 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bbe68-1fe0-4790-9f5e-467f15e853a4 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:55.476174 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c53fcd-84ce-4a83-86b6-58389251a3a2 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:55.477941 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d8c53fcd-84ce-4a83-86b6-58389251a3a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:55.504268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6e1bbe68-1fe0-4790-9f5e-467f15e853a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9fa00514155444489338ff20ecfd862'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.505613 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c53fcd-84ce-4a83-86b6-58389251a3a2 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:55.511526 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd7c0b-1448-4674-b28f-235730cd2fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:55.512548 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd7c0b-1448-4674-b28f-235730cd2fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:55.512867 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd7c0b-1448-4674-b28f-235730cd2fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:55.513146 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd7c0b-1448-4674-b28f-235730cd2fd5 admin admin] PATH_INFO: `/v3/projects/d8d67813ac3a41d6a7b7ae8ac9629d71` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:55.513758 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6e1bbe68-1fe0-4790-9f5e-467f15e853a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9fa00514155444489338ff20ecfd862'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.514392 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd7c0b-1448-4674-b28f-235730cd2fd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8d67813ac3a41d6a7b7ae8ac9629d71)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:55.516447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd7c0b-1448-4674-b28f-235730cd2fd5 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:55.525082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6e1bbe68-1fe0-4790-9f5e-467f15e853a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9fa00514155444489338ff20ecfd862'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.528395 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6e1bbe68-1fe0-4790-9f5e-467f15e853a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9fa00514155444489338ff20ecfd862'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.529205 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-6e1bbe68-1fe0-4790-9f5e-467f15e853a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9fa00514155444489338ff20ecfd862'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.555125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1318/2620] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:41:55 2022] DELETE /identity/v3/users/d9fa00514155444489338ff20ecfd862 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:55.556297 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bbe68-1fe0-4790-9f5e-467f15e853a4 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:55.557770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6e1bbe68-1fe0-4790-9f5e-467f15e853a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:55.576167 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-29fd7c0b-1448-4674-b28f-235730cd2fd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8d67813ac3a41d6a7b7ae8ac9629d71', 'request_id': 'req-29fd7c0b-1448-4674-b28f-235730cd2fd5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.579751 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:55.587384 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-29fd7c0b-1448-4674-b28f-235730cd2fd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8d67813ac3a41d6a7b7ae8ac9629d71', 'request_id': 'req-29fd7c0b-1448-4674-b28f-235730cd2fd5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.593625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:55.596194 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-29fd7c0b-1448-4674-b28f-235730cd2fd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8d67813ac3a41d6a7b7ae8ac9629d71', 'request_id': 'req-29fd7c0b-1448-4674-b28f-235730cd2fd5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.611417 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-29fd7c0b-1448-4674-b28f-235730cd2fd5 admin admin] The token cache is being invalidate because project d8d67813ac3a41d6a7b7ae8ac9629d71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:41:55.611787 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-29fd7c0b-1448-4674-b28f-235730cd2fd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.612374 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-29fd7c0b-1448-4674-b28f-235730cd2fd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.613995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1303/2621] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:41:55 2022] DELETE /identity/v3/projects/d8d67813ac3a41d6a7b7ae8ac9629d71 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:41:55.615435 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07a0d5-78f6-471c-b8e1-c22fd3402cfa None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:55.615720 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07a0d5-78f6-471c-b8e1-c22fd3402cfa None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:55.616068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07a0d5-78f6-471c-b8e1-c22fd3402cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:55.638925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bbe68-1fe0-4790-9f5e-467f15e853a4 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:55.641047 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-dd07a0d5-78f6-471c-b8e1-c22fd3402cfa None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:55.642345 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dd07a0d5-78f6-471c-b8e1-c22fd3402cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:55.700861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c328238-3447-4abd-b403-970621e8f259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:55.701682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c328238-3447-4abd-b403-970621e8f259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:55.701966 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c328238-3447-4abd-b403-970621e8f259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:55.702270 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c328238-3447-4abd-b403-970621e8f259 admin admin] PATH_INFO: `/v3/projects/2df2b1715762427cb824e381a83bdad1` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:55.703535 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c328238-3447-4abd-b403-970621e8f259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2df2b1715762427cb824e381a83bdad1)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:55.705587 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c328238-3447-4abd-b403-970621e8f259 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:55.708261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1304/2622] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:55.709650 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e189b-a819-4163-b530-0cc902777317 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:55.709945 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e189b-a819-4163-b530-0cc902777317 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:55.710233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e189b-a819-4163-b530-0cc902777317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:55.731588 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-193e189b-a819-4163-b530-0cc902777317 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:55.732827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-193e189b-a819-4163-b530-0cc902777317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:55.741881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1305/2623] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:55.742825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e189b-a819-4163-b530-0cc902777317 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:55.749648 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-193e189b-a819-4163-b530-0cc902777317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:55.757236 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-1c328238-3447-4abd-b403-970621e8f259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2df2b1715762427cb824e381a83bdad1', 'request_id': 'req-1c328238-3447-4abd-b403-970621e8f259'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.767874 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-1c328238-3447-4abd-b403-970621e8f259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2df2b1715762427cb824e381a83bdad1', 'request_id': 'req-1c328238-3447-4abd-b403-970621e8f259'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.783076 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-1c328238-3447-4abd-b403-970621e8f259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2df2b1715762427cb824e381a83bdad1', 'request_id': 'req-1c328238-3447-4abd-b403-970621e8f259'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.796949 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-1c328238-3447-4abd-b403-970621e8f259 admin admin] The token cache is being invalidate because project 2df2b1715762427cb824e381a83bdad1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:41:55.797433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-1c328238-3447-4abd-b403-970621e8f259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.798134 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-1c328238-3447-4abd-b403-970621e8f259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.802679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1319/2624] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:41:55 2022] DELETE /identity/v3/projects/2df2b1715762427cb824e381a83bdad1 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:55.802679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc010e-17c5-44c9-8b8a-99d37b219a3f None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:55.802679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc010e-17c5-44c9-8b8a-99d37b219a3f None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:55.802679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc010e-17c5-44c9-8b8a-99d37b219a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:55.822079 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-10fc010e-17c5-44c9-8b8a-99d37b219a3f None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:55.823865 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-10fc010e-17c5-44c9-8b8a-99d37b219a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:55.838986 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e189b-a819-4163-b530-0cc902777317 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:55.893632 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1320/2625] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:55.894622 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc010e-17c5-44c9-8b8a-99d37b219a3f None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:55.896055 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-10fc010e-17c5-44c9-8b8a-99d37b219a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:55.901713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cdd35c-1ff5-4630-bdf1-5d6e31a13fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:55.902376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdd35c-1ff5-4630-bdf1-5d6e31a13fbd service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:55.902667 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdd35c-1ff5-4630-bdf1-5d6e31a13fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:55.902971 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdd35c-1ff5-4630-bdf1-5d6e31a13fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:55.904813 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d5cdd35c-1ff5-4630-bdf1-5d6e31a13fbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:55.907709 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc010e-17c5-44c9-8b8a-99d37b219a3f None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:55.912284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70bbc0-8b80-47d6-8c9c-3388dcaeef43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:55.913154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70bbc0-8b80-47d6-8c9c-3388dcaeef43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:55.913500 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70bbc0-8b80-47d6-8c9c-3388dcaeef43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:55.913705 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70bbc0-8b80-47d6-8c9c-3388dcaeef43 admin admin] PATH_INFO: `/v3/users/14fc83ff7a094a0da889617d7ab2b178` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:55.914902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70bbc0-8b80-47d6-8c9c-3388dcaeef43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14fc83ff7a094a0da889617d7ab2b178)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:55.917399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70bbc0-8b80-47d6-8c9c-3388dcaeef43 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:55.917539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cdd35c-1ff5-4630-bdf1-5d6e31a13fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:55.918992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cdd35c-1ff5-4630-bdf1-5d6e31a13fbd service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:55.919119 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:55.929501 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1306/2626] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:41:55.930326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cdd35c-1ff5-4630-bdf1-5d6e31a13fbd service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:55.941076 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cdd35c-1ff5-4630-bdf1-5d6e31a13fbd service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:55.946581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989ba47-f89d-42c9-9fdd-3077df6cf9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:55.947417 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989ba47-f89d-42c9-9fdd-3077df6cf9cc admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:55.947878 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989ba47-f89d-42c9-9fdd-3077df6cf9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:55.948063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989ba47-f89d-42c9-9fdd-3077df6cf9cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:55.949005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989ba47-f89d-42c9-9fdd-3077df6cf9cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:41:55.949353 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989ba47-f89d-42c9-9fdd-3077df6cf9cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:55.951470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989ba47-f89d-42c9-9fdd-3077df6cf9cc admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:55.969678 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1307/2627] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:41:55.971728 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989ba47-f89d-42c9-9fdd-3077df6cf9cc admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:55.975849 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-5d70bbc0-8b80-47d6-8c9c-3388dcaeef43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14fc83ff7a094a0da889617d7ab2b178'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.984362 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989ba47-f89d-42c9-9fdd-3077df6cf9cc admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:55.986240 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-5d70bbc0-8b80-47d6-8c9c-3388dcaeef43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14fc83ff7a094a0da889617d7ab2b178'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.991725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa238940-2000-4839-83ff-b05c6cdb4d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:55.992273 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa238940-2000-4839-83ff-b05c6cdb4d60 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:55.992736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa238940-2000-4839-83ff-b05c6cdb4d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:55.993135 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa238940-2000-4839-83ff-b05c6cdb4d60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:55.993918 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa238940-2000-4839-83ff-b05c6cdb4d60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:55.996661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-5d70bbc0-8b80-47d6-8c9c-3388dcaeef43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14fc83ff7a094a0da889617d7ab2b178'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.996814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa238940-2000-4839-83ff-b05c6cdb4d60 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:55.997568 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-5d70bbc0-8b80-47d6-8c9c-3388dcaeef43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14fc83ff7a094a0da889617d7ab2b178'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:55.998380 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-5d70bbc0-8b80-47d6-8c9c-3388dcaeef43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14fc83ff7a094a0da889617d7ab2b178'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:56.024310 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:56.024651 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1321/2628] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:41:55 2022] DELETE /identity/v3/users/14fc83ff7a094a0da889617d7ab2b178 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:56.038441 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1308/2629] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:41:55 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:41:56.038966 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70bbc0-8b80-47d6-8c9c-3388dcaeef43 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:56.040455 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-5d70bbc0-8b80-47d6-8c9c-3388dcaeef43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:56.127720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70bbc0-8b80-47d6-8c9c-3388dcaeef43 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:56.187042 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40067550-27e4-4c2e-9bbd-451b630441a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:56.187727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40067550-27e4-4c2e-9bbd-451b630441a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:56.188016 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40067550-27e4-4c2e-9bbd-451b630441a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:56.188296 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40067550-27e4-4c2e-9bbd-451b630441a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:56.189001 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40067550-27e4-4c2e-9bbd-451b630441a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:56.191097 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40067550-27e4-4c2e-9bbd-451b630441a1 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:56.212611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1322/2630] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:41:56 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:41:56.217227 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa238940-2000-4839-83ff-b05c6cdb4d60 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:56.228365 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa238940-2000-4839-83ff-b05c6cdb4d60 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:56.234046 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abbc50a-fcb1-4e45-a6a5-caebcf1b0b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:56.234866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbc50a-fcb1-4e45-a6a5-caebcf1b0b2f admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:56.235437 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbc50a-fcb1-4e45-a6a5-caebcf1b0b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:56.235927 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbc50a-fcb1-4e45-a6a5-caebcf1b0b2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:56.237028 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abbc50a-fcb1-4e45-a6a5-caebcf1b0b2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:56.238820 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abbc50a-fcb1-4e45-a6a5-caebcf1b0b2f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:56.254452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1309/2631] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:56.258614 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40067550-27e4-4c2e-9bbd-451b630441a1 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:56.270499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40067550-27e4-4c2e-9bbd-451b630441a1 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:56.276569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f1028-a35e-458a-a674-a4d91b1db852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:56.277269 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f1028-a35e-458a-a674-a4d91b1db852 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:56.277596 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f1028-a35e-458a-a674-a4d91b1db852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:56.277850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f1028-a35e-458a-a674-a4d91b1db852 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:56.278659 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f1028-a35e-458a-a674-a4d91b1db852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:56.280410 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f1028-a35e-458a-a674-a4d91b1db852 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:56.297157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1323/2632] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:56.299157 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abbc50a-fcb1-4e45-a6a5-caebcf1b0b2f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:56.319386 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abbc50a-fcb1-4e45-a6a5-caebcf1b0b2f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:56.319685 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f1028-a35e-458a-a674-a4d91b1db852 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:56.321960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-597f1028-a35e-458a-a674-a4d91b1db852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:56.326743 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbcdfe-b5a5-42d6-870c-c3ed67e329a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:56.329019 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbcdfe-b5a5-42d6-870c-c3ed67e329a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:56.329648 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbcdfe-b5a5-42d6-870c-c3ed67e329a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:56.330058 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbcdfe-b5a5-42d6-870c-c3ed67e329a8 admin admin] PATH_INFO: `/v3/projects/d560d6eb35a84b818714812740ef84cf/users/5d9d154fe4cf483cabd187c8012a9b13/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:56.350436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:56.357232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f1028-a35e-458a-a674-a4d91b1db852 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:56.360262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbcdfe-b5a5-42d6-870c-c3ed67e329a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d560d6eb35a84b818714812740ef84cf, user_id=5d9d154fe4cf483cabd187c8012a9b13, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:56.363435 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e1809-3b1b-4346-b0c4-288c4277f865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:56.364520 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e1809-3b1b-4346-b0c4-288c4277f865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:56.364981 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e1809-3b1b-4346-b0c4-288c4277f865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:56.365765 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbcdfe-b5a5-42d6-870c-c3ed67e329a8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:56.365936 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e1809-3b1b-4346-b0c4-288c4277f865 admin admin] PATH_INFO: `/v3/projects/3a764b26086c41fc8a4d790eba1ae62a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:56.367666 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e1809-3b1b-4346-b0c4-288c4277f865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a764b26086c41fc8a4d790eba1ae62a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:56.369651 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e1809-3b1b-4346-b0c4-288c4277f865 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:56.384274 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1310/2633] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:41:56 2022] PUT /identity/v3/projects/d560d6eb35a84b818714812740ef84cf/users/5d9d154fe4cf483cabd187c8012a9b13/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:56.385099 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbcdfe-b5a5-42d6-870c-c3ed67e329a8 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:56.387325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5edbcdfe-b5a5-42d6-870c-c3ed67e329a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:56.424539 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-159e1809-3b1b-4346-b0c4-288c4277f865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a764b26086c41fc8a4d790eba1ae62a', 'request_id': 'req-159e1809-3b1b-4346-b0c4-288c4277f865'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:56.433166 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-159e1809-3b1b-4346-b0c4-288c4277f865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a764b26086c41fc8a4d790eba1ae62a', 'request_id': 'req-159e1809-3b1b-4346-b0c4-288c4277f865'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:56.442089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-159e1809-3b1b-4346-b0c4-288c4277f865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a764b26086c41fc8a4d790eba1ae62a', 'request_id': 'req-159e1809-3b1b-4346-b0c4-288c4277f865'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:56.458444 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-159e1809-3b1b-4346-b0c4-288c4277f865 admin admin] The token cache is being invalidate because project 3a764b26086c41fc8a4d790eba1ae62a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:41:56.458750 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-159e1809-3b1b-4346-b0c4-288c4277f865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:56.459766 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-159e1809-3b1b-4346-b0c4-288c4277f865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:56.462435 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1324/2634] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:41:56 2022] DELETE /identity/v3/projects/3a764b26086c41fc8a4d790eba1ae62a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:56.465152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc111c-b256-451c-9cbf-51b8e83a9f97 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:56.465620 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc111c-b256-451c-9cbf-51b8e83a9f97 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:56.466121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc111c-b256-451c-9cbf-51b8e83a9f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:56.481749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbcdfe-b5a5-42d6-870c-c3ed67e329a8 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:56.499483 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-76dc111c-b256-451c-9cbf-51b8e83a9f97 None None] MFA Rules not processed for user `5d9d154fe4cf483cabd187c8012a9b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:56.501089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-76dc111c-b256-451c-9cbf-51b8e83a9f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:56.569500 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57e266-86f7-49c5-b994-fc16cbe877d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:56.572122 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57e266-86f7-49c5-b994-fc16cbe877d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:56.572421 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57e266-86f7-49c5-b994-fc16cbe877d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:56.573326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57e266-86f7-49c5-b994-fc16cbe877d3 admin admin] PATH_INFO: `/v3/projects/db8108a8afa9403aa78475c06496b951` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:56.576092 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57e266-86f7-49c5-b994-fc16cbe877d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db8108a8afa9403aa78475c06496b951)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:56.578554 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57e266-86f7-49c5-b994-fc16cbe877d3 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:56.604985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1325/2635] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:56.628100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc111c-b256-451c-9cbf-51b8e83a9f97 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:56.628100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-76dc111c-b256-451c-9cbf-51b8e83a9f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:56.669769 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7c57e266-86f7-49c5-b994-fc16cbe877d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db8108a8afa9403aa78475c06496b951', 'request_id': 'req-7c57e266-86f7-49c5-b994-fc16cbe877d3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:56.682227 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7c57e266-86f7-49c5-b994-fc16cbe877d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db8108a8afa9403aa78475c06496b951', 'request_id': 'req-7c57e266-86f7-49c5-b994-fc16cbe877d3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:56.689664 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7c57e266-86f7-49c5-b994-fc16cbe877d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db8108a8afa9403aa78475c06496b951', 'request_id': 'req-7c57e266-86f7-49c5-b994-fc16cbe877d3'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:56.702116 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:56.707317 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7c57e266-86f7-49c5-b994-fc16cbe877d3 admin admin] The token cache is being invalidate because project db8108a8afa9403aa78475c06496b951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:41:56.707708 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7c57e266-86f7-49c5-b994-fc16cbe877d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:56.708999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7c57e266-86f7-49c5-b994-fc16cbe877d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:41:56.711558 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1311/2636] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:41:56 2022] DELETE /identity/v3/projects/db8108a8afa9403aa78475c06496b951 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:56.717725 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be3c93-3489-4f93-896b-eab1a01d3753 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:56.718022 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be3c93-3489-4f93-896b-eab1a01d3753 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:56.718451 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be3c93-3489-4f93-896b-eab1a01d3753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:56.738473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-14be3c93-3489-4f93-896b-eab1a01d3753 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:56.739866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-14be3c93-3489-4f93-896b-eab1a01d3753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:56.751345 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc111c-b256-451c-9cbf-51b8e83a9f97 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:56.765671 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:56.811117 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1312/2637] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:56.819337 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e80bf-095b-49f1-af0d-d1d6075af4d5 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:56.819337 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e80bf-095b-49f1-af0d-d1d6075af4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:56.819770 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e80bf-095b-49f1-af0d-d1d6075af4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:56.820416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad7ffa-2f24-4f51-85eb-6d9c6245bbc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:56.821085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad7ffa-2f24-4f51-85eb-6d9c6245bbc4 service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:56.821377 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad7ffa-2f24-4f51-85eb-6d9c6245bbc4 service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:56.821651 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad7ffa-2f24-4f51-85eb-6d9c6245bbc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:56.824451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-40ad7ffa-2f24-4f51-85eb-6d9c6245bbc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:56.839966 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-044e80bf-095b-49f1-af0d-d1d6075af4d5 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:56.841189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-044e80bf-095b-49f1-af0d-d1d6075af4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:56.855290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1313/2638] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:41:56.860120 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e80bf-095b-49f1-af0d-d1d6075af4d5 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:56.872685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e80bf-095b-49f1-af0d-d1d6075af4d5 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:56.877888 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb2a21-0c46-4cb2-97e4-cdcc8d04aff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:56.878544 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb2a21-0c46-4cb2-97e4-cdcc8d04aff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:56.878826 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb2a21-0c46-4cb2-97e4-cdcc8d04aff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:56.879133 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb2a21-0c46-4cb2-97e4-cdcc8d04aff8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:56.880103 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb2a21-0c46-4cb2-97e4-cdcc8d04aff8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:41:56.880408 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb2a21-0c46-4cb2-97e4-cdcc8d04aff8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:56.882002 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb2a21-0c46-4cb2-97e4-cdcc8d04aff8 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:56.888976 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad7ffa-2f24-4f51-85eb-6d9c6245bbc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:56.890580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad7ffa-2f24-4f51-85eb-6d9c6245bbc4 service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:56.894859 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1314/2639] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 20:41:56.899220 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb2a21-0c46-4cb2-97e4-cdcc8d04aff8 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:56.900320 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1326/2640] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:41:56.909729 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb2a21-0c46-4cb2-97e4-cdcc8d04aff8 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:56.914902 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c34d2-95b5-4c0d-82d8-751b037cbbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:56.915247 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c34d2-95b5-4c0d-82d8-751b037cbbf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:56.915657 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c34d2-95b5-4c0d-82d8-751b037cbbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:56.915943 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c34d2-95b5-4c0d-82d8-751b037cbbf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:56.916674 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c34d2-95b5-4c0d-82d8-751b037cbbf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:56.918595 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c34d2-95b5-4c0d-82d8-751b037cbbf7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:56.946821 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1315/2641] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:41:56 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:41:56.951836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad7ffa-2f24-4f51-85eb-6d9c6245bbc4 service nova] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:56.962585 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad7ffa-2f24-4f51-85eb-6d9c6245bbc4 service nova] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:57.024147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a71203-9698-420d-80c1-3c8a071f1c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:57.024980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a71203-9698-420d-80c1-3c8a071f1c18 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:57.025272 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a71203-9698-420d-80c1-3c8a071f1c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:57.025551 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a71203-9698-420d-80c1-3c8a071f1c18 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:57.026476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a71203-9698-420d-80c1-3c8a071f1c18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:57.028428 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a71203-9698-420d-80c1-3c8a071f1c18 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:57.050301 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1327/2642] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:41:56 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:41:57.054982 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c34d2-95b5-4c0d-82d8-751b037cbbf7 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:57.067031 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c34d2-95b5-4c0d-82d8-751b037cbbf7 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:57.074177 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202372c-4035-4a56-bec5-6981bd60e553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:57.074851 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202372c-4035-4a56-bec5-6981bd60e553 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:57.075186 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202372c-4035-4a56-bec5-6981bd60e553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:57.075467 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202372c-4035-4a56-bec5-6981bd60e553 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:57.076203 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7202372c-4035-4a56-bec5-6981bd60e553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:57.077842 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7202372c-4035-4a56-bec5-6981bd60e553 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:57.088831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1316/2643] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:57.093689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a71203-9698-420d-80c1-3c8a071f1c18 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:57.105364 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a71203-9698-420d-80c1-3c8a071f1c18 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:57.110444 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee7a18-7522-4b60-bb1f-806d3d5449a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:57.111151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee7a18-7522-4b60-bb1f-806d3d5449a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:57.111434 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee7a18-7522-4b60-bb1f-806d3d5449a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:57.111819 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee7a18-7522-4b60-bb1f-806d3d5449a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:57.112690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ee7a18-7522-4b60-bb1f-806d3d5449a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:57.114734 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ee7a18-7522-4b60-bb1f-806d3d5449a9 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:57.128218 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1328/2644] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:41:57.132722 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202372c-4035-4a56-bec5-6981bd60e553 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:57.144506 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202372c-4035-4a56-bec5-6981bd60e553 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:57.150622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e249745-a46c-4238-9ff5-2b779b920e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:57.151602 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e249745-a46c-4238-9ff5-2b779b920e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:57.151893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e249745-a46c-4238-9ff5-2b779b920e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:57.152155 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e249745-a46c-4238-9ff5-2b779b920e5e admin admin] PATH_INFO: `/v3/projects/af15ec2c218c487aac99bf8a5975e18c/users/d28dff565c2843059ccfc3fcd9da158a/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:57.173272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e249745-a46c-4238-9ff5-2b779b920e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=af15ec2c218c487aac99bf8a5975e18c, user_id=d28dff565c2843059ccfc3fcd9da158a, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:57.176344 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e249745-a46c-4238-9ff5-2b779b920e5e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:57.191325 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1317/2645] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:41:57 2022] PUT /identity/v3/projects/af15ec2c218c487aac99bf8a5975e18c/users/d28dff565c2843059ccfc3fcd9da158a/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:41:57.201133 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc6bc5-bc03-4db9-a51d-5e3551d38a15 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:57.201432 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc6bc5-bc03-4db9-a51d-5e3551d38a15 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:57.201715 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc6bc5-bc03-4db9-a51d-5e3551d38a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:57.237192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-66fc6bc5-bc03-4db9-a51d-5e3551d38a15 None None] MFA Rules not processed for user `d28dff565c2843059ccfc3fcd9da158a`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:41:57.238486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-66fc6bc5-bc03-4db9-a51d-5e3551d38a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:57.261281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:41:57.323070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1329/2646] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:41:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:41:57.333796 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e249745-a46c-4238-9ff5-2b779b920e5e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:41:57.335621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-3e249745-a46c-4238-9ff5-2b779b920e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:41:57.394857 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e249745-a46c-4238-9ff5-2b779b920e5e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:41:57.398553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74094aad-301e-4474-8522-f3590cdfb657 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:41:57.399234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74094aad-301e-4474-8522-f3590cdfb657 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:41:57.399519 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74094aad-301e-4474-8522-f3590cdfb657 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:41:57.399789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74094aad-301e-4474-8522-f3590cdfb657 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:41:57.408123 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74094aad-301e-4474-8522-f3590cdfb657 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:41:57.409545 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74094aad-301e-4474-8522-f3590cdfb657 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:41:57.419068 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1318/2647] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:41:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:42:02.522951 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc6bc5-bc03-4db9-a51d-5e3551d38a15 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:02.589510 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc6bc5-bc03-4db9-a51d-5e3551d38a15 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:02.594341 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821dd5b-d0b8-4ce4-95b5-7fe0085be514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:02.595284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821dd5b-d0b8-4ce4-95b5-7fe0085be514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:02.595576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821dd5b-d0b8-4ce4-95b5-7fe0085be514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:02.595852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821dd5b-d0b8-4ce4-95b5-7fe0085be514 admin admin] PATH_INFO: `/v3/users/d28dff565c2843059ccfc3fcd9da158a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:02.597391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821dd5b-d0b8-4ce4-95b5-7fe0085be514 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d28dff565c2843059ccfc3fcd9da158a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:02.599509 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821dd5b-d0b8-4ce4-95b5-7fe0085be514 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:02.651746 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0821dd5b-d0b8-4ce4-95b5-7fe0085be514 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd28dff565c2843059ccfc3fcd9da158a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:02.659497 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0821dd5b-d0b8-4ce4-95b5-7fe0085be514 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd28dff565c2843059ccfc3fcd9da158a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:02.667902 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0821dd5b-d0b8-4ce4-95b5-7fe0085be514 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd28dff565c2843059ccfc3fcd9da158a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:02.668612 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0821dd5b-d0b8-4ce4-95b5-7fe0085be514 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd28dff565c2843059ccfc3fcd9da158a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:02.669226 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-0821dd5b-d0b8-4ce4-95b5-7fe0085be514 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd28dff565c2843059ccfc3fcd9da158a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:02.688509 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1330/2648] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:42:02 2022] DELETE /identity/v3/users/d28dff565c2843059ccfc3fcd9da158a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:02.694637 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936ea70-be01-49bf-a6d7-b3a83763423b None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:02.695365 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936ea70-be01-49bf-a6d7-b3a83763423b None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:02.695745 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936ea70-be01-49bf-a6d7-b3a83763423b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:02.716503 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f936ea70-be01-49bf-a6d7-b3a83763423b None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:02.717832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f936ea70-be01-49bf-a6d7-b3a83763423b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:02.738264 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:02.777622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1319/2649] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:02.787373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821dd5b-d0b8-4ce4-95b5-7fe0085be514 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:02.789032 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-0821dd5b-d0b8-4ce4-95b5-7fe0085be514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:02.836070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:02.852609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821dd5b-d0b8-4ce4-95b5-7fe0085be514 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:02.856856 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4b45f-8f04-4f83-bda9-49ec8b064e46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:02.857503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4b45f-8f04-4f83-bda9-49ec8b064e46 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:02.857783 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4b45f-8f04-4f83-bda9-49ec8b064e46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:02.858060 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4b45f-8f04-4f83-bda9-49ec8b064e46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:02.867510 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4b45f-8f04-4f83-bda9-49ec8b064e46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:02.869417 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4b45f-8f04-4f83-bda9-49ec8b064e46 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:02.879786 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1331/2650] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:42:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:42:03.160829 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936ea70-be01-49bf-a6d7-b3a83763423b None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:03.162260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f936ea70-be01-49bf-a6d7-b3a83763423b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:03.178678 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936ea70-be01-49bf-a6d7-b3a83763423b None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:03.183375 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e63b1-50eb-41e4-b42f-54148d19bddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:03.184443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e63b1-50eb-41e4-b42f-54148d19bddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:03.184549 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e63b1-50eb-41e4-b42f-54148d19bddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:03.184759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e63b1-50eb-41e4-b42f-54148d19bddd admin admin] PATH_INFO: `/v3/projects/af15ec2c218c487aac99bf8a5975e18c` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:03.185940 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e63b1-50eb-41e4-b42f-54148d19bddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=af15ec2c218c487aac99bf8a5975e18c)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:03.187872 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e63b1-50eb-41e4-b42f-54148d19bddd admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:03.235928 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bc0e63b1-50eb-41e4-b42f-54148d19bddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af15ec2c218c487aac99bf8a5975e18c', 'request_id': 'req-bc0e63b1-50eb-41e4-b42f-54148d19bddd'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:03.242216 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bc0e63b1-50eb-41e4-b42f-54148d19bddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af15ec2c218c487aac99bf8a5975e18c', 'request_id': 'req-bc0e63b1-50eb-41e4-b42f-54148d19bddd'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:03.248847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bc0e63b1-50eb-41e4-b42f-54148d19bddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af15ec2c218c487aac99bf8a5975e18c', 'request_id': 'req-bc0e63b1-50eb-41e4-b42f-54148d19bddd'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:03.261429 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bc0e63b1-50eb-41e4-b42f-54148d19bddd admin admin] The token cache is being invalidate because project af15ec2c218c487aac99bf8a5975e18c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:42:03.261718 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bc0e63b1-50eb-41e4-b42f-54148d19bddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:03.262332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bc0e63b1-50eb-41e4-b42f-54148d19bddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:03.263944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1320/2651] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:42:03 2022] DELETE /identity/v3/projects/af15ec2c218c487aac99bf8a5975e18c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:03.269747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883560fa-b73a-4baf-85f6-99364ac10238 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:03.270057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883560fa-b73a-4baf-85f6-99364ac10238 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:03.270338 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883560fa-b73a-4baf-85f6-99364ac10238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:03.288056 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-883560fa-b73a-4baf-85f6-99364ac10238 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:03.289192 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-883560fa-b73a-4baf-85f6-99364ac10238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:03.352839 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1332/2652] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:03.361856 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e3830-0c29-4b9d-9a20-612bb40785f5 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:03.362172 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e3830-0c29-4b9d-9a20-612bb40785f5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:03.362450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e3830-0c29-4b9d-9a20-612bb40785f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:03.380881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-9e1e3830-0c29-4b9d-9a20-612bb40785f5 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:03.382095 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9e1e3830-0c29-4b9d-9a20-612bb40785f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:03.390444 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1321/2653] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:03.394968 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883560fa-b73a-4baf-85f6-99364ac10238 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:03.404818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883560fa-b73a-4baf-85f6-99364ac10238 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:03.409410 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bbcfd-a058-4825-964c-2519bd0b9a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:03.410064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bbcfd-a058-4825-964c-2519bd0b9a88 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:03.410349 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bbcfd-a058-4825-964c-2519bd0b9a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:03.410629 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bbcfd-a058-4825-964c-2519bd0b9a88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:03.411419 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bbcfd-a058-4825-964c-2519bd0b9a88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:42:03.411752 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bbcfd-a058-4825-964c-2519bd0b9a88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:03.413333 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bbcfd-a058-4825-964c-2519bd0b9a88 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:03.428700 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1333/2654] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:42:03.430017 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e3830-0c29-4b9d-9a20-612bb40785f5 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:03.440032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e3830-0c29-4b9d-9a20-612bb40785f5 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:03.444882 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1320a21-8e9f-4bd0-b927-fe12bef26021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:03.445530 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1320a21-8e9f-4bd0-b927-fe12bef26021 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:03.445814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1320a21-8e9f-4bd0-b927-fe12bef26021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:03.446118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1320a21-8e9f-4bd0-b927-fe12bef26021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:03.446810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1320a21-8e9f-4bd0-b927-fe12bef26021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:03.448761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1320a21-8e9f-4bd0-b927-fe12bef26021 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:03.474578 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1322/2655] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:42:03 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:42:03.479535 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bbcfd-a058-4825-964c-2519bd0b9a88 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:03.489552 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bbcfd-a058-4825-964c-2519bd0b9a88 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:03.550695 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85defa79-c38f-4087-8e6e-c29834f71507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:03.551489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85defa79-c38f-4087-8e6e-c29834f71507 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:03.551620 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85defa79-c38f-4087-8e6e-c29834f71507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:03.551835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85defa79-c38f-4087-8e6e-c29834f71507 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:03.552523 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85defa79-c38f-4087-8e6e-c29834f71507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:03.554447 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85defa79-c38f-4087-8e6e-c29834f71507 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:03.589108 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1334/2656] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:42:03 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:42:03.593308 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1320a21-8e9f-4bd0-b927-fe12bef26021 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:03.604893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1320a21-8e9f-4bd0-b927-fe12bef26021 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:03.609700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb861af-258d-472d-a1d5-2f5b788a787d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:03.610384 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb861af-258d-472d-a1d5-2f5b788a787d admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:03.610664 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb861af-258d-472d-a1d5-2f5b788a787d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:03.610929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb861af-258d-472d-a1d5-2f5b788a787d admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:03.611689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb861af-258d-472d-a1d5-2f5b788a787d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:03.613276 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb861af-258d-472d-a1d5-2f5b788a787d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:03.624759 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1323/2657] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:42:03.629051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85defa79-c38f-4087-8e6e-c29834f71507 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:03.640708 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85defa79-c38f-4087-8e6e-c29834f71507 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:03.651720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa916a4-2b1d-4e8b-87b3-d79d3117fe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:03.652334 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa916a4-2b1d-4e8b-87b3-d79d3117fe60 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:03.652547 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa916a4-2b1d-4e8b-87b3-d79d3117fe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:03.652829 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa916a4-2b1d-4e8b-87b3-d79d3117fe60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:03.654060 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa916a4-2b1d-4e8b-87b3-d79d3117fe60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:03.655639 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa916a4-2b1d-4e8b-87b3-d79d3117fe60 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:03.668052 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1335/2658] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:42:03.672996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb861af-258d-472d-a1d5-2f5b788a787d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:03.684184 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb861af-258d-472d-a1d5-2f5b788a787d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:03.690251 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3e087-2eee-4175-a453-63ef4750c4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:03.691191 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3e087-2eee-4175-a453-63ef4750c4a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:03.691467 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3e087-2eee-4175-a453-63ef4750c4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:03.691738 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3e087-2eee-4175-a453-63ef4750c4a1 admin admin] PATH_INFO: `/v3/projects/c8ded6a71283444e892094501f29e4da/users/c0d43152eeb9485e9ff16f38aa63ac22/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:03.707799 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a3e087-2eee-4175-a453-63ef4750c4a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ded6a71283444e892094501f29e4da, user_id=c0d43152eeb9485e9ff16f38aa63ac22, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:03.712602 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a3e087-2eee-4175-a453-63ef4750c4a1 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:03.730701 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1324/2659] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:42:03 2022] PUT /identity/v3/projects/c8ded6a71283444e892094501f29e4da/users/c0d43152eeb9485e9ff16f38aa63ac22/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:42:03.737131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3be657-05da-4712-ba22-e78a0964b0b8 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:03.737436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3be657-05da-4712-ba22-e78a0964b0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:03.737746 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3be657-05da-4712-ba22-e78a0964b0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:03.767990 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-9a3be657-05da-4712-ba22-e78a0964b0b8 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:03.770247 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9a3be657-05da-4712-ba22-e78a0964b0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:03.844657 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:03.864347 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1336/2660] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:42:03.874591 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3e087-2eee-4175-a453-63ef4750c4a1 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:03.876342 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-28a3e087-2eee-4175-a453-63ef4750c4a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:03.931164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:03.940682 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3e087-2eee-4175-a453-63ef4750c4a1 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:03.945566 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d58622-896c-4c79-9119-8fedaf21e667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:03.946228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d58622-896c-4c79-9119-8fedaf21e667 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:03.946514 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d58622-896c-4c79-9119-8fedaf21e667 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:03.946791 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d58622-896c-4c79-9119-8fedaf21e667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:03.957666 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d58622-896c-4c79-9119-8fedaf21e667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:03.958990 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d58622-896c-4c79-9119-8fedaf21e667 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:03.972881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1325/2661] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:42:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:42:04.254636 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3be657-05da-4712-ba22-e78a0964b0b8 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:04.254636 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9a3be657-05da-4712-ba22-e78a0964b0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:04.267486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3be657-05da-4712-ba22-e78a0964b0b8 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:04.272850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d258f-7833-4101-8d28-6198997de14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:04.273766 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d258f-7833-4101-8d28-6198997de14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:04.274127 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d258f-7833-4101-8d28-6198997de14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:04.274311 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d258f-7833-4101-8d28-6198997de14d admin admin] PATH_INFO: `/v3/users/5d9d154fe4cf483cabd187c8012a9b13` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:04.275618 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d258f-7833-4101-8d28-6198997de14d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d9d154fe4cf483cabd187c8012a9b13)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:04.282163 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d258f-7833-4101-8d28-6198997de14d admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:04.354115 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-4e2d258f-7833-4101-8d28-6198997de14d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d9d154fe4cf483cabd187c8012a9b13'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:04.363563 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-4e2d258f-7833-4101-8d28-6198997de14d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d9d154fe4cf483cabd187c8012a9b13'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:04.376358 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-4e2d258f-7833-4101-8d28-6198997de14d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d9d154fe4cf483cabd187c8012a9b13'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:04.377888 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-4e2d258f-7833-4101-8d28-6198997de14d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d9d154fe4cf483cabd187c8012a9b13'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:04.378712 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-4e2d258f-7833-4101-8d28-6198997de14d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d9d154fe4cf483cabd187c8012a9b13'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:04.410793 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1337/2662] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:42:04 2022] DELETE /identity/v3/users/5d9d154fe4cf483cabd187c8012a9b13 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:04.417355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055dbc7-280e-45e9-a606-c9b3e00aeca7 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:04.418137 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055dbc7-280e-45e9-a606-c9b3e00aeca7 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:04.418604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055dbc7-280e-45e9-a606-c9b3e00aeca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:04.440793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4055dbc7-280e-45e9-a606-c9b3e00aeca7 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:04.442029 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4055dbc7-280e-45e9-a606-c9b3e00aeca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:04.487830 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d258f-7833-4101-8d28-6198997de14d admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:04.489600 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4e2d258f-7833-4101-8d28-6198997de14d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:04.513476 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1326/2663] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:04.524575 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055dbc7-280e-45e9-a606-c9b3e00aeca7 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:04.526908 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4055dbc7-280e-45e9-a606-c9b3e00aeca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:04.559997 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d258f-7833-4101-8d28-6198997de14d admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:04.564888 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7f68c-4aa8-4b38-8fa4-63938ffc7bcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:04.565608 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7f68c-4aa8-4b38-8fa4-63938ffc7bcc service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:04.565959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7f68c-4aa8-4b38-8fa4-63938ffc7bcc service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:04.566238 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7f68c-4aa8-4b38-8fa4-63938ffc7bcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:04.568121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-57d7f68c-4aa8-4b38-8fa4-63938ffc7bcc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:04.603273 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055dbc7-280e-45e9-a606-c9b3e00aeca7 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:04.608991 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e3e3f-e0be-4246-9561-98683a0cf139 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:04.609981 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e3e3f-e0be-4246-9561-98683a0cf139 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:04.610343 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e3e3f-e0be-4246-9561-98683a0cf139 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:04.611128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e3e3f-e0be-4246-9561-98683a0cf139 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:04.621506 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e3e3f-e0be-4246-9561-98683a0cf139 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:04.623261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e3e3f-e0be-4246-9561-98683a0cf139 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:04.647520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1327/2664] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:42:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:42:04.655960 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7f68c-4aa8-4b38-8fa4-63938ffc7bcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:04.657517 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7f68c-4aa8-4b38-8fa4-63938ffc7bcc service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:04.675594 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1338/2665] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:42:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:42:05.239387 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e3e3f-e0be-4246-9561-98683a0cf139 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:05.241623 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-680e3e3f-e0be-4246-9561-98683a0cf139 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:05.255398 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e3e3f-e0be-4246-9561-98683a0cf139 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:05.262209 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a3176-93c4-46d1-924a-71bc2769f6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:05.263148 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a3176-93c4-46d1-924a-71bc2769f6ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:05.264639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a3176-93c4-46d1-924a-71bc2769f6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:05.264639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a3176-93c4-46d1-924a-71bc2769f6ac admin admin] PATH_INFO: `/v3/projects/d560d6eb35a84b818714812740ef84cf` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:05.265266 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a3176-93c4-46d1-924a-71bc2769f6ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=d560d6eb35a84b818714812740ef84cf)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:05.278374 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a3176-93c4-46d1-924a-71bc2769f6ac admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:05.359282 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-765a3176-93c4-46d1-924a-71bc2769f6ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd560d6eb35a84b818714812740ef84cf', 'request_id': 'req-765a3176-93c4-46d1-924a-71bc2769f6ac'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:05.371196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-765a3176-93c4-46d1-924a-71bc2769f6ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd560d6eb35a84b818714812740ef84cf', 'request_id': 'req-765a3176-93c4-46d1-924a-71bc2769f6ac'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:05.380999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-765a3176-93c4-46d1-924a-71bc2769f6ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd560d6eb35a84b818714812740ef84cf', 'request_id': 'req-765a3176-93c4-46d1-924a-71bc2769f6ac'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:05.401147 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-765a3176-93c4-46d1-924a-71bc2769f6ac admin admin] The token cache is being invalidate because project d560d6eb35a84b818714812740ef84cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:42:05.401458 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-765a3176-93c4-46d1-924a-71bc2769f6ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:05.402766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-765a3176-93c4-46d1-924a-71bc2769f6ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:05.408405 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1328/2666] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:42:05 2022] DELETE /identity/v3/projects/d560d6eb35a84b818714812740ef84cf => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:05.409816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868e6ad-035a-485c-a7e8-4b54ff776084 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:05.410136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868e6ad-035a-485c-a7e8-4b54ff776084 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:05.410424 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868e6ad-035a-485c-a7e8-4b54ff776084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:05.430972 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-1868e6ad-035a-485c-a7e8-4b54ff776084 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:05.434548 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-1868e6ad-035a-485c-a7e8-4b54ff776084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:05.448650 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:05.522916 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1339/2667] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:05.532151 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd7b87-09f8-4ea8-81ef-6d06b3acfaf6 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:05.532755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd7b87-09f8-4ea8-81ef-6d06b3acfaf6 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:05.533049 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd7b87-09f8-4ea8-81ef-6d06b3acfaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:05.555441 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-fbdd7b87-09f8-4ea8-81ef-6d06b3acfaf6 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:05.556721 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-fbdd7b87-09f8-4ea8-81ef-6d06b3acfaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:05.567241 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1329/2668] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:05.572091 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868e6ad-035a-485c-a7e8-4b54ff776084 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:05.588143 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868e6ad-035a-485c-a7e8-4b54ff776084 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:05.594297 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf9aee-dd28-4b02-92e2-7cc30f1461ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:05.595015 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf9aee-dd28-4b02-92e2-7cc30f1461ee admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:05.595301 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf9aee-dd28-4b02-92e2-7cc30f1461ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:05.595629 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf9aee-dd28-4b02-92e2-7cc30f1461ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:05.596505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf9aee-dd28-4b02-92e2-7cc30f1461ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:42:05.596827 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf9aee-dd28-4b02-92e2-7cc30f1461ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:05.598698 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf9aee-dd28-4b02-92e2-7cc30f1461ee admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:05.619089 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1340/2669] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:42:05.620313 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd7b87-09f8-4ea8-81ef-6d06b3acfaf6 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:05.633957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd7b87-09f8-4ea8-81ef-6d06b3acfaf6 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:05.646063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335877e-82d7-4479-a90b-d834a8b78c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:05.647494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335877e-82d7-4479-a90b-d834a8b78c96 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:05.647965 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335877e-82d7-4479-a90b-d834a8b78c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:05.648258 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335877e-82d7-4479-a90b-d834a8b78c96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:05.649052 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335877e-82d7-4479-a90b-d834a8b78c96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:05.651341 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335877e-82d7-4479-a90b-d834a8b78c96 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:05.660213 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:05.711980 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1330/2670] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:42:05 2022] POST /identity/v3/projects => generated 405 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:42:05.716788 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf9aee-dd28-4b02-92e2-7cc30f1461ee admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:05.729596 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf9aee-dd28-4b02-92e2-7cc30f1461ee admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:05.803559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05d755-282d-473a-b4b0-2721c36cba49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:05.804267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05d755-282d-473a-b4b0-2721c36cba49 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:05.804557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05d755-282d-473a-b4b0-2721c36cba49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:05.804838 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05d755-282d-473a-b4b0-2721c36cba49 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:05.805556 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05d755-282d-473a-b4b0-2721c36cba49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:05.807637 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05d755-282d-473a-b4b0-2721c36cba49 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:05.840742 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1341/2671] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:42:05 2022] POST /identity/v3/users => generated 303 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:42:05.842064 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335877e-82d7-4479-a90b-d834a8b78c96 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:05.858913 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335877e-82d7-4479-a90b-d834a8b78c96 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:05.864795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7139b11-d092-44d1-a19c-b91fca165c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:05.864795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7139b11-d092-44d1-a19c-b91fca165c6c admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:05.864795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7139b11-d092-44d1-a19c-b91fca165c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:05.865195 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7139b11-d092-44d1-a19c-b91fca165c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:05.873070 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7139b11-d092-44d1-a19c-b91fca165c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:05.873070 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7139b11-d092-44d1-a19c-b91fca165c6c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:05.895915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1331/2672] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:42:05.899778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05d755-282d-473a-b4b0-2721c36cba49 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:05.911077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05d755-282d-473a-b4b0-2721c36cba49 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:05.916227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08554476-b1d9-4662-b319-755a35ab5691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:05.917433 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08554476-b1d9-4662-b319-755a35ab5691 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:05.917724 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08554476-b1d9-4662-b319-755a35ab5691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:05.918009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08554476-b1d9-4662-b319-755a35ab5691 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:05.918751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08554476-b1d9-4662-b319-755a35ab5691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:05.922074 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08554476-b1d9-4662-b319-755a35ab5691 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:05.937723 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1342/2673] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:42:05.942543 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7139b11-d092-44d1-a19c-b91fca165c6c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:05.953946 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7139b11-d092-44d1-a19c-b91fca165c6c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:05.959038 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78be2ad-b180-4180-9a10-5805441934f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:05.960149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78be2ad-b180-4180-9a10-5805441934f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:05.960505 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78be2ad-b180-4180-9a10-5805441934f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:05.960874 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78be2ad-b180-4180-9a10-5805441934f9 admin admin] PATH_INFO: `/v3/projects/d9301219ff9f401eb25df8ae3a933072/users/2f307994063f47e8a4547c480a85262f/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:05.979302 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78be2ad-b180-4180-9a10-5805441934f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9301219ff9f401eb25df8ae3a933072, user_id=2f307994063f47e8a4547c480a85262f, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:05.982279 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78be2ad-b180-4180-9a10-5805441934f9 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:06.003967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1332/2674] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:42:05 2022] PUT /identity/v3/projects/d9301219ff9f401eb25df8ae3a933072/users/2f307994063f47e8a4547c480a85262f/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:06.014279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad72887-cb7f-4b19-97be-cddb8715e9d4 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:06.014648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad72887-cb7f-4b19-97be-cddb8715e9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:06.015025 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad72887-cb7f-4b19-97be-cddb8715e9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:06.053369 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-0ad72887-cb7f-4b19-97be-cddb8715e9d4 None None] MFA Rules not processed for user `2f307994063f47e8a4547c480a85262f`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:06.054680 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-0ad72887-cb7f-4b19-97be-cddb8715e9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:06.137590 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1343/2675] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:06 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:06.142364 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78be2ad-b180-4180-9a10-5805441934f9 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:06.212298 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78be2ad-b180-4180-9a10-5805441934f9 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:06.218463 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4c1d8-6525-4176-b18b-f46084e976f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:06.219280 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4c1d8-6525-4176-b18b-f46084e976f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:06.219460 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4c1d8-6525-4176-b18b-f46084e976f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:06.219882 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4c1d8-6525-4176-b18b-f46084e976f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:06.220609 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4c1d8-6525-4176-b18b-f46084e976f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:06.222761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4c1d8-6525-4176-b18b-f46084e976f1 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:06.252138 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1333/2676] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:42:06 2022] POST /identity/v3/projects => generated 405 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:42:06.256225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad72887-cb7f-4b19-97be-cddb8715e9d4 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:06.275285 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad72887-cb7f-4b19-97be-cddb8715e9d4 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:06.289173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:06.353965 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfddda-7a30-4609-8774-7f148fbc3cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:06.354692 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfddda-7a30-4609-8774-7f148fbc3cae admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:06.355051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfddda-7a30-4609-8774-7f148fbc3cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:06.355941 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfddda-7a30-4609-8774-7f148fbc3cae admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:06.356081 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cfddda-7a30-4609-8774-7f148fbc3cae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:06.358005 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cfddda-7a30-4609-8774-7f148fbc3cae admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:06.380928 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1344/2677] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:42:06 2022] POST /identity/v3/users => generated 309 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:42:06.384714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4c1d8-6525-4176-b18b-f46084e976f1 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:06.396970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4c1d8-6525-4176-b18b-f46084e976f1 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:06.402452 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6873bf-fd8b-4575-bdff-bb9c1957f7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:06.403569 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6873bf-fd8b-4575-bdff-bb9c1957f7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:06.404006 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6873bf-fd8b-4575-bdff-bb9c1957f7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:06.404582 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6873bf-fd8b-4575-bdff-bb9c1957f7f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:06.405480 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6873bf-fd8b-4575-bdff-bb9c1957f7f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:06.407128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6873bf-fd8b-4575-bdff-bb9c1957f7f6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:06.418774 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1334/2678] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:42:06.423184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfddda-7a30-4609-8774-7f148fbc3cae admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:06.438281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfddda-7a30-4609-8774-7f148fbc3cae admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:06.443977 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38378dd4-2575-464d-9c9b-2c7cf837e048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:06.445133 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38378dd4-2575-464d-9c9b-2c7cf837e048 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:06.445489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38378dd4-2575-464d-9c9b-2c7cf837e048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:06.445836 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38378dd4-2575-464d-9c9b-2c7cf837e048 admin admin] PATH_INFO: `/v3/projects/ab7d1a1f1ebd49a7b6bd91c1f3802a9c/users/456402212d9a486ab5359b05d4d64dae/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:06.471769 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38378dd4-2575-464d-9c9b-2c7cf837e048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7d1a1f1ebd49a7b6bd91c1f3802a9c, user_id=456402212d9a486ab5359b05d4d64dae, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:06.474741 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38378dd4-2575-464d-9c9b-2c7cf837e048 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:06.507449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1345/2679] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:42:06 2022] PUT /identity/v3/projects/ab7d1a1f1ebd49a7b6bd91c1f3802a9c/users/456402212d9a486ab5359b05d4d64dae/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:06.511244 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6873bf-fd8b-4575-bdff-bb9c1957f7f6 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:06.532756 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6873bf-fd8b-4575-bdff-bb9c1957f7f6 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:06.629425 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befe7de-7f07-4071-b306-8e404652bcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:06.630144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befe7de-7f07-4071-b306-8e404652bcda admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:06.631970 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befe7de-7f07-4071-b306-8e404652bcda admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:06.632260 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befe7de-7f07-4071-b306-8e404652bcda admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:06.633036 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befe7de-7f07-4071-b306-8e404652bcda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:06.634706 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befe7de-7f07-4071-b306-8e404652bcda admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:06.656040 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1335/2680] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:42:06.659734 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38378dd4-2575-464d-9c9b-2c7cf837e048 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:06.673171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38378dd4-2575-464d-9c9b-2c7cf837e048 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:06.679370 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e2d68-af96-49fc-81a6-c460a051cb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:06.680645 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e2d68-af96-49fc-81a6-c460a051cb7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:06.681037 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e2d68-af96-49fc-81a6-c460a051cb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:06.681550 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e2d68-af96-49fc-81a6-c460a051cb7f admin admin] PATH_INFO: `/v3/projects/ab7d1a1f1ebd49a7b6bd91c1f3802a9c/users/456402212d9a486ab5359b05d4d64dae/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:06.684990 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e2d68-af96-49fc-81a6-c460a051cb7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7d1a1f1ebd49a7b6bd91c1f3802a9c, user_id=456402212d9a486ab5359b05d4d64dae, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:06.688064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e2d68-af96-49fc-81a6-c460a051cb7f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:06.711064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1346/2681] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:42:06 2022] PUT /identity/v3/projects/ab7d1a1f1ebd49a7b6bd91c1f3802a9c/users/456402212d9a486ab5359b05d4d64dae/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:06.717790 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcb52d-b0a0-44ad-9dcd-867d8cdfed5e None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:06.718510 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcb52d-b0a0-44ad-9dcd-867d8cdfed5e None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:06.719038 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcb52d-b0a0-44ad-9dcd-867d8cdfed5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:06.755999 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-e7bcb52d-b0a0-44ad-9dcd-867d8cdfed5e None None] MFA Rules not processed for user `456402212d9a486ab5359b05d4d64dae`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:06.757357 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e7bcb52d-b0a0-44ad-9dcd-867d8cdfed5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:06.768482 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:06.868763 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1336/2682] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:06 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:42:06.878666 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e2d68-af96-49fc-81a6-c460a051cb7f admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:06.880645 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-678e2d68-af96-49fc-81a6-c460a051cb7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:06.950815 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e2d68-af96-49fc-81a6-c460a051cb7f admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:06.954557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a577b9-caad-4144-aafa-f46eae7da202 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:06.955246 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a577b9-caad-4144-aafa-f46eae7da202 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:06.955528 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a577b9-caad-4144-aafa-f46eae7da202 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:06.955805 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a577b9-caad-4144-aafa-f46eae7da202 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:06.966552 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a577b9-caad-4144-aafa-f46eae7da202 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:06.968117 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a577b9-caad-4144-aafa-f46eae7da202 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:06.979733 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1347/2683] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:42:06 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:42:07.844128 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bcb52d-b0a0-44ad-9dcd-867d8cdfed5e None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:07.948257 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bcb52d-b0a0-44ad-9dcd-867d8cdfed5e None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:07.959189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79c767-d139-4505-8497-74082ac83242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:07.960281 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79c767-d139-4505-8497-74082ac83242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:07.960571 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79c767-d139-4505-8497-74082ac83242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:07.960853 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79c767-d139-4505-8497-74082ac83242 admin admin] PATH_INFO: `/v3/users/2f307994063f47e8a4547c480a85262f` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:07.963354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79c767-d139-4505-8497-74082ac83242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f307994063f47e8a4547c480a85262f)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:07.966402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79c767-d139-4505-8497-74082ac83242 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:08.031750 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7f79c767-d139-4505-8497-74082ac83242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f307994063f47e8a4547c480a85262f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:08.046466 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7f79c767-d139-4505-8497-74082ac83242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f307994063f47e8a4547c480a85262f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:08.047205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7f79c767-d139-4505-8497-74082ac83242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f307994063f47e8a4547c480a85262f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:08.047957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7f79c767-d139-4505-8497-74082ac83242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f307994063f47e8a4547c480a85262f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:08.066305 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7f79c767-d139-4505-8497-74082ac83242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f307994063f47e8a4547c480a85262f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:08.082316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1337/2684] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:42:07 2022] DELETE /identity/v3/users/2f307994063f47e8a4547c480a85262f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:08.086173 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3da46-8bcf-464e-b199-daab92e1600f None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:08.086538 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3da46-8bcf-464e-b199-daab92e1600f None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:08.086634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3da46-8bcf-464e-b199-daab92e1600f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:08.110844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-57c3da46-8bcf-464e-b199-daab92e1600f None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:08.112374 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-57c3da46-8bcf-464e-b199-daab92e1600f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:08.216942 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1348/2685] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:08.228025 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79c767-d139-4505-8497-74082ac83242 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:08.229513 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7f79c767-d139-4505-8497-74082ac83242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:08.282497 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:08.347349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79c767-d139-4505-8497-74082ac83242 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:08.353442 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf60811-c83a-4473-a73c-b8b79d43ef28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:08.356594 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf60811-c83a-4473-a73c-b8b79d43ef28 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:08.357195 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf60811-c83a-4473-a73c-b8b79d43ef28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:08.358304 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf60811-c83a-4473-a73c-b8b79d43ef28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:08.379887 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf60811-c83a-4473-a73c-b8b79d43ef28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:08.381465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf60811-c83a-4473-a73c-b8b79d43ef28 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:08.417779 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1338/2686] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:42:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:42:08.808467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3da46-8bcf-464e-b199-daab92e1600f None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:08.809953 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-57c3da46-8bcf-464e-b199-daab92e1600f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:08.835499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3da46-8bcf-464e-b199-daab92e1600f None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:08.837684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c409259-c889-4a1f-908e-6b6dc3a18876 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:08.838556 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c409259-c889-4a1f-908e-6b6dc3a18876 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:08.838791 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c409259-c889-4a1f-908e-6b6dc3a18876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:08.847861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d885a75-1434-4c7f-aea2-38766f987c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:08.858449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d885a75-1434-4c7f-aea2-38766f987c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:08.859051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d885a75-1434-4c7f-aea2-38766f987c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:08.859351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d885a75-1434-4c7f-aea2-38766f987c9d admin admin] PATH_INFO: `/v3/projects/d9301219ff9f401eb25df8ae3a933072` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:08.863109 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d885a75-1434-4c7f-aea2-38766f987c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9301219ff9f401eb25df8ae3a933072)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:08.866542 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d885a75-1434-4c7f-aea2-38766f987c9d admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:08.879383 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:08.912636 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4c409259-c889-4a1f-908e-6b6dc3a18876 None None] MFA Rules not processed for user `c0d43152eeb9485e9ff16f38aa63ac22`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:08.914483 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4c409259-c889-4a1f-908e-6b6dc3a18876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:08.963412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3d885a75-1434-4c7f-aea2-38766f987c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9301219ff9f401eb25df8ae3a933072', 'request_id': 'req-3d885a75-1434-4c7f-aea2-38766f987c9d'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:08.973284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3d885a75-1434-4c7f-aea2-38766f987c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9301219ff9f401eb25df8ae3a933072', 'request_id': 'req-3d885a75-1434-4c7f-aea2-38766f987c9d'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:08.982395 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3d885a75-1434-4c7f-aea2-38766f987c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9301219ff9f401eb25df8ae3a933072', 'request_id': 'req-3d885a75-1434-4c7f-aea2-38766f987c9d'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:08.995864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3d885a75-1434-4c7f-aea2-38766f987c9d admin admin] The token cache is being invalidate because project d9301219ff9f401eb25df8ae3a933072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:42:08.996447 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3d885a75-1434-4c7f-aea2-38766f987c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:08.997274 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3d885a75-1434-4c7f-aea2-38766f987c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:08.999033 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1349/2687] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:42:08 2022] DELETE /identity/v3/projects/d9301219ff9f401eb25df8ae3a933072 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:09.003051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d885a75-1434-4c7f-aea2-38766f987c9d admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:09.004496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3d885a75-1434-4c7f-aea2-38766f987c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:09.006081 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1339/2688] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:42:09.011036 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c409259-c889-4a1f-908e-6b6dc3a18876 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:09.012868 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4c409259-c889-4a1f-908e-6b6dc3a18876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:09.029124 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c409259-c889-4a1f-908e-6b6dc3a18876 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:09.093328 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d885a75-1434-4c7f-aea2-38766f987c9d admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:09.096007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455e137-5ae6-4d5d-a122-40744ea1e584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:09.096880 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455e137-5ae6-4d5d-a122-40744ea1e584 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:09.097271 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455e137-5ae6-4d5d-a122-40744ea1e584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:09.097456 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455e137-5ae6-4d5d-a122-40744ea1e584 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:09.098166 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455e137-5ae6-4d5d-a122-40744ea1e584 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:09.098351 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb0d2b-bb4b-4f3e-8202-d27bb9d8e2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:09.099473 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb0d2b-bb4b-4f3e-8202-d27bb9d8e2a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:09.099754 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb0d2b-bb4b-4f3e-8202-d27bb9d8e2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:09.100026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb0d2b-bb4b-4f3e-8202-d27bb9d8e2a9 admin admin] PATH_INFO: `/v3/users/456402212d9a486ab5359b05d4d64dae` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:09.100272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455e137-5ae6-4d5d-a122-40744ea1e584 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:09.101445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb0d2b-bb4b-4f3e-8202-d27bb9d8e2a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=456402212d9a486ab5359b05d4d64dae)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:09.104175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb0d2b-bb4b-4f3e-8202-d27bb9d8e2a9 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:09.131800 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1340/2689] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:42:09 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:42:09.136731 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455e137-5ae6-4d5d-a122-40744ea1e584 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:09.138451 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-6455e137-5ae6-4d5d-a122-40744ea1e584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:09.162334 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-78eb0d2b-bb4b-4f3e-8202-d27bb9d8e2a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '456402212d9a486ab5359b05d4d64dae'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:09.171284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-78eb0d2b-bb4b-4f3e-8202-d27bb9d8e2a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '456402212d9a486ab5359b05d4d64dae'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:09.181099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-78eb0d2b-bb4b-4f3e-8202-d27bb9d8e2a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '456402212d9a486ab5359b05d4d64dae'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:09.181948 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-78eb0d2b-bb4b-4f3e-8202-d27bb9d8e2a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '456402212d9a486ab5359b05d4d64dae'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:09.182587 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-78eb0d2b-bb4b-4f3e-8202-d27bb9d8e2a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '456402212d9a486ab5359b05d4d64dae'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:09.202323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:09.205910 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1350/2690] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:42:09 2022] DELETE /identity/v3/users/456402212d9a486ab5359b05d4d64dae => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:42:09.225323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455e137-5ae6-4d5d-a122-40744ea1e584 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:09.291367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da951633-731d-4b85-9555-7f30d5529b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:09.292054 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da951633-731d-4b85-9555-7f30d5529b83 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:09.292339 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da951633-731d-4b85-9555-7f30d5529b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:09.292614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da951633-731d-4b85-9555-7f30d5529b83 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:09.293463 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da951633-731d-4b85-9555-7f30d5529b83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:09.295890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da951633-731d-4b85-9555-7f30d5529b83 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:09.321703 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1341/2691] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:42:09 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:42:09.323760 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb0d2b-bb4b-4f3e-8202-d27bb9d8e2a9 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:09.325300 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-78eb0d2b-bb4b-4f3e-8202-d27bb9d8e2a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:09.342421 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb0d2b-bb4b-4f3e-8202-d27bb9d8e2a9 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:09.347353 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f3eee-85a9-4df0-ace2-fa7a2fab44ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:09.348043 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f3eee-85a9-4df0-ace2-fa7a2fab44ef admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:09.348330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f3eee-85a9-4df0-ace2-fa7a2fab44ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:09.348614 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f3eee-85a9-4df0-ace2-fa7a2fab44ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:09.349363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f3eee-85a9-4df0-ace2-fa7a2fab44ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:09.351065 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f3eee-85a9-4df0-ace2-fa7a2fab44ef admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:09.358666 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:09.365312 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1351/2692] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:42:09.366924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da951633-731d-4b85-9555-7f30d5529b83 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:09.379132 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da951633-731d-4b85-9555-7f30d5529b83 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:09.384296 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03ee48-65ff-4084-b28c-610bda9280c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:09.385715 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03ee48-65ff-4084-b28c-610bda9280c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:09.385992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03ee48-65ff-4084-b28c-610bda9280c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:09.386259 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03ee48-65ff-4084-b28c-610bda9280c4 admin admin] PATH_INFO: `/v3/projects/47ebe372bc944ef7967dca5e2ba857f1/users/aab62a5b981d4aadbd48cd751c5bb529/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:09.404030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03ee48-65ff-4084-b28c-610bda9280c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ebe372bc944ef7967dca5e2ba857f1, user_id=aab62a5b981d4aadbd48cd751c5bb529, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:09.406709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03ee48-65ff-4084-b28c-610bda9280c4 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:09.422860 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1342/2693] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:42:09 2022] PUT /identity/v3/projects/47ebe372bc944ef7967dca5e2ba857f1/users/aab62a5b981d4aadbd48cd751c5bb529/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:09.426912 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f3eee-85a9-4df0-ace2-fa7a2fab44ef admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:09.440504 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f3eee-85a9-4df0-ace2-fa7a2fab44ef admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:09.511392 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594fe23-cca2-452a-afd3-b8924edb9d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:09.512080 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594fe23-cca2-452a-afd3-b8924edb9d78 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:09.512535 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594fe23-cca2-452a-afd3-b8924edb9d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:09.512827 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594fe23-cca2-452a-afd3-b8924edb9d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:09.513588 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2594fe23-cca2-452a-afd3-b8924edb9d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:09.515969 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2594fe23-cca2-452a-afd3-b8924edb9d78 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:09.529388 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03ee48-65ff-4084-b28c-610bda9280c4 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:09.534399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1352/2694] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:42:09.535537 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-5a03ee48-65ff-4084-b28c-610bda9280c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:09.541876 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594fe23-cca2-452a-afd3-b8924edb9d78 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:09.556202 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03ee48-65ff-4084-b28c-610bda9280c4 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:09.561263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94a5a3-c217-4dbf-abab-3faac8e06c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:09.562126 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94a5a3-c217-4dbf-abab-3faac8e06c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:09.562404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94a5a3-c217-4dbf-abab-3faac8e06c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:09.562703 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94a5a3-c217-4dbf-abab-3faac8e06c9f admin admin] PATH_INFO: `/v3/projects/ab7d1a1f1ebd49a7b6bd91c1f3802a9c` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:09.563978 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94a5a3-c217-4dbf-abab-3faac8e06c9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab7d1a1f1ebd49a7b6bd91c1f3802a9c)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:09.565176 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594fe23-cca2-452a-afd3-b8924edb9d78 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:09.565883 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94a5a3-c217-4dbf-abab-3faac8e06c9f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:09.572384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf84b10-b7b9-4d7a-981e-ac51aaa7f740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:09.573225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf84b10-b7b9-4d7a-981e-ac51aaa7f740 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:09.573886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf84b10-b7b9-4d7a-981e-ac51aaa7f740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:09.574078 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf84b10-b7b9-4d7a-981e-ac51aaa7f740 admin admin] PATH_INFO: `/v3/projects/47ebe372bc944ef7967dca5e2ba857f1/users/aab62a5b981d4aadbd48cd751c5bb529/roles/5ca112885467422684e134cb0f94865a` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:09.576233 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf84b10-b7b9-4d7a-981e-ac51aaa7f740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ebe372bc944ef7967dca5e2ba857f1, user_id=aab62a5b981d4aadbd48cd751c5bb529, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:09.580301 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf84b10-b7b9-4d7a-981e-ac51aaa7f740 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:09.593867 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1353/2695] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:42:09 2022] PUT /identity/v3/projects/47ebe372bc944ef7967dca5e2ba857f1/users/aab62a5b981d4aadbd48cd751c5bb529/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:09.619878 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7f94a5a3-c217-4dbf-abab-3faac8e06c9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab7d1a1f1ebd49a7b6bd91c1f3802a9c', 'request_id': 'req-7f94a5a3-c217-4dbf-abab-3faac8e06c9f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:09.632624 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7f94a5a3-c217-4dbf-abab-3faac8e06c9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab7d1a1f1ebd49a7b6bd91c1f3802a9c', 'request_id': 'req-7f94a5a3-c217-4dbf-abab-3faac8e06c9f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:09.640044 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7f94a5a3-c217-4dbf-abab-3faac8e06c9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab7d1a1f1ebd49a7b6bd91c1f3802a9c', 'request_id': 'req-7f94a5a3-c217-4dbf-abab-3faac8e06c9f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:09.654815 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7f94a5a3-c217-4dbf-abab-3faac8e06c9f admin admin] The token cache is being invalidate because project ab7d1a1f1ebd49a7b6bd91c1f3802a9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:42:09.655224 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7f94a5a3-c217-4dbf-abab-3faac8e06c9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:09.655933 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7f94a5a3-c217-4dbf-abab-3faac8e06c9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:09.661444 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1343/2696] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:42:09 2022] DELETE /identity/v3/projects/ab7d1a1f1ebd49a7b6bd91c1f3802a9c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:42:09.663255 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c02842-71d3-4b45-95a3-ed98e79f0005 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:09.663634 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c02842-71d3-4b45-95a3-ed98e79f0005 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:09.663918 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c02842-71d3-4b45-95a3-ed98e79f0005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:09.682731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-a4c02842-71d3-4b45-95a3-ed98e79f0005 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:09.684011 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-a4c02842-71d3-4b45-95a3-ed98e79f0005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:09.770787 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1354/2697] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:09.778766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd10cb9-dfd1-4b93-9b2b-5217e683dd77 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:09.779167 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd10cb9-dfd1-4b93-9b2b-5217e683dd77 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:09.779581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd10cb9-dfd1-4b93-9b2b-5217e683dd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:09.809034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-1fd10cb9-dfd1-4b93-9b2b-5217e683dd77 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:09.810274 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1fd10cb9-dfd1-4b93-9b2b-5217e683dd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:09.819785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1344/2698] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:09.824580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c02842-71d3-4b45-95a3-ed98e79f0005 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:09.837213 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c02842-71d3-4b45-95a3-ed98e79f0005 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:09.842169 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987cb97-347b-4800-ba65-57908f7cb94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:09.843070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987cb97-347b-4800-ba65-57908f7cb94f admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:09.843208 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987cb97-347b-4800-ba65-57908f7cb94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:09.843426 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987cb97-347b-4800-ba65-57908f7cb94f admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:09.844202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987cb97-347b-4800-ba65-57908f7cb94f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:42:09.844518 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987cb97-347b-4800-ba65-57908f7cb94f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:09.846154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987cb97-347b-4800-ba65-57908f7cb94f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:09.859997 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1355/2699] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:42:09.864685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd10cb9-dfd1-4b93-9b2b-5217e683dd77 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:09.876278 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd10cb9-dfd1-4b93-9b2b-5217e683dd77 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:09.881656 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc45f6-f618-4b8b-9710-0b97e4ff691c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:09.882349 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc45f6-f618-4b8b-9710-0b97e4ff691c admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:09.882629 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc45f6-f618-4b8b-9710-0b97e4ff691c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:09.882898 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc45f6-f618-4b8b-9710-0b97e4ff691c admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:09.884055 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc45f6-f618-4b8b-9710-0b97e4ff691c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:09.892622 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc45f6-f618-4b8b-9710-0b97e4ff691c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:09.947030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1345/2700] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:42:09 2022] POST /identity/v3/projects => generated 401 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:42:09.948348 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987cb97-347b-4800-ba65-57908f7cb94f admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:09.962929 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987cb97-347b-4800-ba65-57908f7cb94f admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:10.035285 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744bbc9-f158-4ed0-b61b-2c6fb187942f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:10.036312 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744bbc9-f158-4ed0-b61b-2c6fb187942f admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:10.036822 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744bbc9-f158-4ed0-b61b-2c6fb187942f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:10.037722 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744bbc9-f158-4ed0-b61b-2c6fb187942f admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:10.038773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744bbc9-f158-4ed0-b61b-2c6fb187942f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:10.042854 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744bbc9-f158-4ed0-b61b-2c6fb187942f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:10.063561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:10.075086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1356/2701] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:42:09 2022] POST /identity/v3/users => generated 301 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:42:10.078996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc45f6-f618-4b8b-9710-0b97e4ff691c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:10.098069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc45f6-f618-4b8b-9710-0b97e4ff691c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:10.105390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ba865-3dd8-4a27-9892-1f41faa7dbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:10.106253 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ba865-3dd8-4a27-9892-1f41faa7dbd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:10.106551 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ba865-3dd8-4a27-9892-1f41faa7dbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:10.106804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ba865-3dd8-4a27-9892-1f41faa7dbd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:10.107582 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ba865-3dd8-4a27-9892-1f41faa7dbd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:10.109237 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ba865-3dd8-4a27-9892-1f41faa7dbd9 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:10.117149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:10.130016 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1346/2702] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:42:10.130552 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744bbc9-f158-4ed0-b61b-2c6fb187942f admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:10.145167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744bbc9-f158-4ed0-b61b-2c6fb187942f admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:10.150254 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efa193-3d44-44dd-b91c-8727f584a0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:10.151167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efa193-3d44-44dd-b91c-8727f584a0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:10.151551 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efa193-3d44-44dd-b91c-8727f584a0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:10.152222 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efa193-3d44-44dd-b91c-8727f584a0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:10.152761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efa193-3d44-44dd-b91c-8727f584a0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:10.154707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efa193-3d44-44dd-b91c-8727f584a0e6 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:10.167800 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1357/2703] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:42:10.171872 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ba865-3dd8-4a27-9892-1f41faa7dbd9 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:10.184035 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ba865-3dd8-4a27-9892-1f41faa7dbd9 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:10.189367 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb45e95-b5d5-42a3-b5f4-49d89a68f9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:10.190361 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb45e95-b5d5-42a3-b5f4-49d89a68f9a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:10.190675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb45e95-b5d5-42a3-b5f4-49d89a68f9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:10.191032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb45e95-b5d5-42a3-b5f4-49d89a68f9a7 admin admin] PATH_INFO: `/v3/projects/d8130deded584e99a426b49e949e033a/users/78b3d9a0ddf8493681c4a763a3d07b1c/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:10.206995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb45e95-b5d5-42a3-b5f4-49d89a68f9a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8130deded584e99a426b49e949e033a, user_id=78b3d9a0ddf8493681c4a763a3d07b1c, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:10.209672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb45e95-b5d5-42a3-b5f4-49d89a68f9a7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:10.225043 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1347/2704] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:42:10 2022] PUT /identity/v3/projects/d8130deded584e99a426b49e949e033a/users/78b3d9a0ddf8493681c4a763a3d07b1c/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:10.231064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ba49e-48b7-4948-8904-ff62a4d6d875 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:10.231484 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ba49e-48b7-4948-8904-ff62a4d6d875 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:10.231775 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ba49e-48b7-4948-8904-ff62a4d6d875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:10.251256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-cb9ba49e-48b7-4948-8904-ff62a4d6d875 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:10.252168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-cb9ba49e-48b7-4948-8904-ff62a4d6d875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:10.336777 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1358/2705] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:10.349766 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb45e95-b5d5-42a3-b5f4-49d89a68f9a7 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:10.351681 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7bb45e95-b5d5-42a3-b5f4-49d89a68f9a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:10.437932 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb45e95-b5d5-42a3-b5f4-49d89a68f9a7 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:10.441948 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580de4c0-c9fe-49b5-b00b-0637cc8ed875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:10.442814 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580de4c0-c9fe-49b5-b00b-0637cc8ed875 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:10.443076 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580de4c0-c9fe-49b5-b00b-0637cc8ed875 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:10.443356 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580de4c0-c9fe-49b5-b00b-0637cc8ed875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:10.457005 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580de4c0-c9fe-49b5-b00b-0637cc8ed875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:10.458481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580de4c0-c9fe-49b5-b00b-0637cc8ed875 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:10.469952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1348/2706] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:42:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:42:11.727129 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfe00a-660e-45bb-92eb-483d90818dd1 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:11.727588 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfe00a-660e-45bb-92eb-483d90818dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:11.727684 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfe00a-660e-45bb-92eb-483d90818dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:11.799384 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-00dfe00a-660e-45bb-92eb-483d90818dd1 None None] MFA Rules not processed for user `78b3d9a0ddf8493681c4a763a3d07b1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:11.801072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-00dfe00a-660e-45bb-92eb-483d90818dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:12.037096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1359/2707] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:12.053311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580de4c0-c9fe-49b5-b00b-0637cc8ed875 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:12.059580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44ecdb-46b3-4a18-8e95-8ecccf57cb85 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:12.059873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44ecdb-46b3-4a18-8e95-8ecccf57cb85 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:12.060161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44ecdb-46b3-4a18-8e95-8ecccf57cb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:12.083569 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-4b44ecdb-46b3-4a18-8e95-8ecccf57cb85 None None] MFA Rules not processed for user `6502e3ffa3b84333a00ea5f712767d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:12.085131 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4b44ecdb-46b3-4a18-8e95-8ecccf57cb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:12.097904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580de4c0-c9fe-49b5-b00b-0637cc8ed875 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:12.111671 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9608e4b-989d-4f9c-9b81-eec83e2376c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:12.112390 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9608e4b-989d-4f9c-9b81-eec83e2376c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:12.112668 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9608e4b-989d-4f9c-9b81-eec83e2376c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:12.112951 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9608e4b-989d-4f9c-9b81-eec83e2376c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:12.113680 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9608e4b-989d-4f9c-9b81-eec83e2376c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:12.115918 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9608e4b-989d-4f9c-9b81-eec83e2376c2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:12.181857 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1349/2708] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:42:12 2022] POST /identity/v3/projects => generated 403 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:42:12.184001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9608e4b-989d-4f9c-9b81-eec83e2376c2 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:12.230564 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1360/2709] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:42:12.289858 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9608e4b-989d-4f9c-9b81-eec83e2376c2 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:12.300412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815603d5-0655-4631-b3c1-1c8e7b35b189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:12.300626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815603d5-0655-4631-b3c1-1c8e7b35b189 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:12.300870 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815603d5-0655-4631-b3c1-1c8e7b35b189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:12.301149 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815603d5-0655-4631-b3c1-1c8e7b35b189 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:12.301856 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815603d5-0655-4631-b3c1-1c8e7b35b189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:12.304216 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815603d5-0655-4631-b3c1-1c8e7b35b189 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:12.364118 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1350/2710] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:42:12 2022] POST /identity/v3/users => generated 308 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:42:12.365472 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44ecdb-46b3-4a18-8e95-8ecccf57cb85 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:12.383167 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44ecdb-46b3-4a18-8e95-8ecccf57cb85 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:12.391530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf9d11-e4d4-425b-a842-60ce920024f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:12.392185 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf9d11-e4d4-425b-a842-60ce920024f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:12.392614 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf9d11-e4d4-425b-a842-60ce920024f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:12.393412 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf9d11-e4d4-425b-a842-60ce920024f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:12.394197 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf9d11-e4d4-425b-a842-60ce920024f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:12.397873 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf9d11-e4d4-425b-a842-60ce920024f4 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:12.428606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1361/2711] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:42:12.433097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815603d5-0655-4631-b3c1-1c8e7b35b189 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:12.452786 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815603d5-0655-4631-b3c1-1c8e7b35b189 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:12.470279 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6866b-59ab-4ae4-b1ed-8c52036ceff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:12.471234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6866b-59ab-4ae4-b1ed-8c52036ceff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:12.471568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6866b-59ab-4ae4-b1ed-8c52036ceff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:12.471847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6866b-59ab-4ae4-b1ed-8c52036ceff0 admin admin] PATH_INFO: `/v3/projects/daba16cd7a9f422ba39ebcc7e61d95cf/users/c7f4e0d8c7074390a861e15f6660ffc4/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:12.511574 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6866b-59ab-4ae4-b1ed-8c52036ceff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daba16cd7a9f422ba39ebcc7e61d95cf, user_id=c7f4e0d8c7074390a861e15f6660ffc4, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:12.514320 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6866b-59ab-4ae4-b1ed-8c52036ceff0 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:12.539140 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1351/2712] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:42:12 2022] PUT /identity/v3/projects/daba16cd7a9f422ba39ebcc7e61d95cf/users/c7f4e0d8c7074390a861e15f6660ffc4/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:12.540926 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf9d11-e4d4-425b-a842-60ce920024f4 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:12.554220 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf9d11-e4d4-425b-a842-60ce920024f4 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:12.618635 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:12.738325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c61375-c9db-42db-b7db-b69d02667de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:12.739221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c61375-c9db-42db-b7db-b69d02667de2 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:12.739580 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c61375-c9db-42db-b7db-b69d02667de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:12.740038 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c61375-c9db-42db-b7db-b69d02667de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:12.740816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c61375-c9db-42db-b7db-b69d02667de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:12.742417 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c61375-c9db-42db-b7db-b69d02667de2 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:12.755285 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1362/2713] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:42:12.760024 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6866b-59ab-4ae4-b1ed-8c52036ceff0 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:12.774511 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6866b-59ab-4ae4-b1ed-8c52036ceff0 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:12.780612 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ca8c6-4ae3-4529-8981-d2b8455599c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:12.781529 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ca8c6-4ae3-4529-8981-d2b8455599c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:12.781819 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ca8c6-4ae3-4529-8981-d2b8455599c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:12.782097 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ca8c6-4ae3-4529-8981-d2b8455599c9 admin admin] PATH_INFO: `/v3/projects/daba16cd7a9f422ba39ebcc7e61d95cf/users/c7f4e0d8c7074390a861e15f6660ffc4/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:12.784291 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ca8c6-4ae3-4529-8981-d2b8455599c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daba16cd7a9f422ba39ebcc7e61d95cf, user_id=c7f4e0d8c7074390a861e15f6660ffc4, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:12.787168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ca8c6-4ae3-4529-8981-d2b8455599c9 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:12.807774 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1352/2714] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:42:12 2022] PUT /identity/v3/projects/daba16cd7a9f422ba39ebcc7e61d95cf/users/c7f4e0d8c7074390a861e15f6660ffc4/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:42:13.831426 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34918a-4767-4aab-93db-353abc5144e6 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:13.832457 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34918a-4767-4aab-93db-353abc5144e6 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:13.832897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34918a-4767-4aab-93db-353abc5144e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:13.890863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-fc34918a-4767-4aab-93db-353abc5144e6 None None] MFA Rules not processed for user `c7f4e0d8c7074390a861e15f6660ffc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:13.892591 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-fc34918a-4767-4aab-93db-353abc5144e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:14.113770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1363/2715] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:13 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:16.439857 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ca8c6-4ae3-4529-8981-d2b8455599c9 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:16.455969 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:16.572184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebe871-23ee-4417-8712-00cf690be2c0 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:16.572701 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebe871-23ee-4417-8712-00cf690be2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:16.572701 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebe871-23ee-4417-8712-00cf690be2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:16.590401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ca8c6-4ae3-4529-8981-d2b8455599c9 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:16.596566 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9828f11-6c9b-4b34-a1c1-d7a246d83890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:16.597738 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9828f11-6c9b-4b34-a1c1-d7a246d83890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:16.598185 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9828f11-6c9b-4b34-a1c1-d7a246d83890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:16.598831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9828f11-6c9b-4b34-a1c1-d7a246d83890 admin admin] PATH_INFO: `/v3/users/78b3d9a0ddf8493681c4a763a3d07b1c` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:16.600559 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9828f11-6c9b-4b34-a1c1-d7a246d83890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b3d9a0ddf8493681c4a763a3d07b1c)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:16.603234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9828f11-6c9b-4b34-a1c1-d7a246d83890 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:16.607761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-f7ebe871-23ee-4417-8712-00cf690be2c0 None None] MFA Rules not processed for user `aab62a5b981d4aadbd48cd751c5bb529`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:16.609101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f7ebe871-23ee-4417-8712-00cf690be2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:16.685702 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a9828f11-6c9b-4b34-a1c1-d7a246d83890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b3d9a0ddf8493681c4a763a3d07b1c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:16.698382 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a9828f11-6c9b-4b34-a1c1-d7a246d83890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b3d9a0ddf8493681c4a763a3d07b1c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:16.700315 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a9828f11-6c9b-4b34-a1c1-d7a246d83890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b3d9a0ddf8493681c4a763a3d07b1c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:16.700315 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a9828f11-6c9b-4b34-a1c1-d7a246d83890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b3d9a0ddf8493681c4a763a3d07b1c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:16.722864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a9828f11-6c9b-4b34-a1c1-d7a246d83890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b3d9a0ddf8493681c4a763a3d07b1c'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:16.727557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1364/2716] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:16 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:16.738396 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebe871-23ee-4417-8712-00cf690be2c0 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:16.740522 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f7ebe871-23ee-4417-8712-00cf690be2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:16.741985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1353/2717] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:42:16 2022] DELETE /identity/v3/users/78b3d9a0ddf8493681c4a763a3d07b1c => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:16.785527 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:16.917539 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebe871-23ee-4417-8712-00cf690be2c0 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:17.007309 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9828f11-6c9b-4b34-a1c1-d7a246d83890 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:17.008950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a9828f11-6c9b-4b34-a1c1-d7a246d83890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:17.020515 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dccc2e-4cd5-4e56-a460-775564d9213d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:17.023926 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dccc2e-4cd5-4e56-a460-775564d9213d service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:17.024696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dccc2e-4cd5-4e56-a460-775564d9213d service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:17.024696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dccc2e-4cd5-4e56-a460-775564d9213d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:17.027344 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f4dccc2e-4cd5-4e56-a460-775564d9213d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:17.113418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dccc2e-4cd5-4e56-a460-775564d9213d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:17.115072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dccc2e-4cd5-4e56-a460-775564d9213d service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:17.127123 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1365/2718] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:42:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:42:17.129355 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9828f11-6c9b-4b34-a1c1-d7a246d83890 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:17.134166 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcea6f-8f60-4593-9d36-dea5866ae383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:17.135209 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcea6f-8f60-4593-9d36-dea5866ae383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:17.135494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcea6f-8f60-4593-9d36-dea5866ae383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:17.135849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcea6f-8f60-4593-9d36-dea5866ae383 admin admin] PATH_INFO: `/v3/projects/d8130deded584e99a426b49e949e033a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:17.137486 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcea6f-8f60-4593-9d36-dea5866ae383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8130deded584e99a426b49e949e033a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:17.139188 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcea6f-8f60-4593-9d36-dea5866ae383 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:17.179550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:17.206263 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aedcea6f-8f60-4593-9d36-dea5866ae383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8130deded584e99a426b49e949e033a', 'request_id': 'req-aedcea6f-8f60-4593-9d36-dea5866ae383'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:17.215571 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aedcea6f-8f60-4593-9d36-dea5866ae383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8130deded584e99a426b49e949e033a', 'request_id': 'req-aedcea6f-8f60-4593-9d36-dea5866ae383'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:17.226100 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aedcea6f-8f60-4593-9d36-dea5866ae383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8130deded584e99a426b49e949e033a', 'request_id': 'req-aedcea6f-8f60-4593-9d36-dea5866ae383'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:17.243555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aedcea6f-8f60-4593-9d36-dea5866ae383 admin admin] The token cache is being invalidate because project d8130deded584e99a426b49e949e033a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:42:17.243845 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aedcea6f-8f60-4593-9d36-dea5866ae383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:17.244595 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aedcea6f-8f60-4593-9d36-dea5866ae383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:17.249105 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1354/2719] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:42:17 2022] DELETE /identity/v3/projects/d8130deded584e99a426b49e949e033a => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:42:17.250318 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dccc2e-4cd5-4e56-a460-775564d9213d service nova] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:17.261618 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dccc2e-4cd5-4e56-a460-775564d9213d service nova] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:17.334467 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:17.351325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5512cfd-f2a6-4ce5-8004-ed0bb1328db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:17.352281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5512cfd-f2a6-4ce5-8004-ed0bb1328db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:17.352571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5512cfd-f2a6-4ce5-8004-ed0bb1328db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:17.352852 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5512cfd-f2a6-4ce5-8004-ed0bb1328db7 admin admin] PATH_INFO: `/v3/users/c7f4e0d8c7074390a861e15f6660ffc4` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:17.372701 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5512cfd-f2a6-4ce5-8004-ed0bb1328db7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7f4e0d8c7074390a861e15f6660ffc4)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:17.374731 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5512cfd-f2a6-4ce5-8004-ed0bb1328db7 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:17.437345 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d5512cfd-f2a6-4ce5-8004-ed0bb1328db7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7f4e0d8c7074390a861e15f6660ffc4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:17.445985 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d5512cfd-f2a6-4ce5-8004-ed0bb1328db7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7f4e0d8c7074390a861e15f6660ffc4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:17.455026 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d5512cfd-f2a6-4ce5-8004-ed0bb1328db7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7f4e0d8c7074390a861e15f6660ffc4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:17.456685 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d5512cfd-f2a6-4ce5-8004-ed0bb1328db7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7f4e0d8c7074390a861e15f6660ffc4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:17.457871 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-d5512cfd-f2a6-4ce5-8004-ed0bb1328db7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7f4e0d8c7074390a861e15f6660ffc4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:17.481070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1366/2720] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:42:17 2022] DELETE /identity/v3/users/c7f4e0d8c7074390a861e15f6660ffc4 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:42:17.754581 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcea6f-8f60-4593-9d36-dea5866ae383 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:17.756480 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-aedcea6f-8f60-4593-9d36-dea5866ae383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:17.796894 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcea6f-8f60-4593-9d36-dea5866ae383 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:17.977888 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4455c-ce2d-4678-b801-a3a08bcc0e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:17.978857 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4455c-ce2d-4678-b801-a3a08bcc0e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:17.979195 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4455c-ce2d-4678-b801-a3a08bcc0e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:17.979554 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4455c-ce2d-4678-b801-a3a08bcc0e7b admin admin] PATH_INFO: `/v3/projects/daba16cd7a9f422ba39ebcc7e61d95cf` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:17.980678 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4455c-ce2d-4678-b801-a3a08bcc0e7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=daba16cd7a9f422ba39ebcc7e61d95cf)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:17.982819 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4455c-ce2d-4678-b801-a3a08bcc0e7b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:18.083037 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c1f4455c-ce2d-4678-b801-a3a08bcc0e7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daba16cd7a9f422ba39ebcc7e61d95cf', 'request_id': 'req-c1f4455c-ce2d-4678-b801-a3a08bcc0e7b'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:18.095450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c1f4455c-ce2d-4678-b801-a3a08bcc0e7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daba16cd7a9f422ba39ebcc7e61d95cf', 'request_id': 'req-c1f4455c-ce2d-4678-b801-a3a08bcc0e7b'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:18.104076 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c1f4455c-ce2d-4678-b801-a3a08bcc0e7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daba16cd7a9f422ba39ebcc7e61d95cf', 'request_id': 'req-c1f4455c-ce2d-4678-b801-a3a08bcc0e7b'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:18.121163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c1f4455c-ce2d-4678-b801-a3a08bcc0e7b admin admin] The token cache is being invalidate because project daba16cd7a9f422ba39ebcc7e61d95cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:42:18.121669 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c1f4455c-ce2d-4678-b801-a3a08bcc0e7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:18.122864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-c1f4455c-ce2d-4678-b801-a3a08bcc0e7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:18.125017 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1355/2721] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:42:17 2022] DELETE /identity/v3/projects/daba16cd7a9f422ba39ebcc7e61d95cf => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:18.130891 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01f4a9-5126-4870-983f-7445ced806a4 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:18.131462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01f4a9-5126-4870-983f-7445ced806a4 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:18.131882 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01f4a9-5126-4870-983f-7445ced806a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:18.156405 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-aa01f4a9-5126-4870-983f-7445ced806a4 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:18.158273 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-aa01f4a9-5126-4870-983f-7445ced806a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:18.237348 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1367/2722] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:18.244256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d6f8c-161e-441b-adb7-e042ebc15700 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:18.244686 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d6f8c-161e-441b-adb7-e042ebc15700 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:18.245917 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d6f8c-161e-441b-adb7-e042ebc15700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:18.276419 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-039d6f8c-161e-441b-adb7-e042ebc15700 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:18.278678 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-039d6f8c-161e-441b-adb7-e042ebc15700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:18.289195 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1356/2723] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:18.293814 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01f4a9-5126-4870-983f-7445ced806a4 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:18.306183 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01f4a9-5126-4870-983f-7445ced806a4 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:18.311823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e2843-6ede-4100-9808-321e4d020de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:18.312739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e2843-6ede-4100-9808-321e4d020de3 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:18.313136 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e2843-6ede-4100-9808-321e4d020de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:18.313606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e2843-6ede-4100-9808-321e4d020de3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:18.314685 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e2843-6ede-4100-9808-321e4d020de3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:42:18.315150 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e2843-6ede-4100-9808-321e4d020de3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:18.316952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e2843-6ede-4100-9808-321e4d020de3 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:18.346491 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1368/2724] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:42:18.350767 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d6f8c-161e-441b-adb7-e042ebc15700 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:18.363763 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d6f8c-161e-441b-adb7-e042ebc15700 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:18.374879 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1b130-d79e-4906-a768-050ba249350c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:18.377989 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1b130-d79e-4906-a768-050ba249350c admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:18.378392 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1b130-d79e-4906-a768-050ba249350c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:18.378775 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1b130-d79e-4906-a768-050ba249350c admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:18.379620 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1b130-d79e-4906-a768-050ba249350c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:18.384189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1b130-d79e-4906-a768-050ba249350c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:18.427605 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1357/2725] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:42:18 2022] POST /identity/v3/projects => generated 369 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:42:18.432335 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e2843-6ede-4100-9808-321e4d020de3 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:18.448469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e2843-6ede-4100-9808-321e4d020de3 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:18.538809 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc9d22-4f5a-45c2-a2a8-d905f15c672a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:18.539523 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc9d22-4f5a-45c2-a2a8-d905f15c672a admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:18.539813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc9d22-4f5a-45c2-a2a8-d905f15c672a admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:18.540093 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc9d22-4f5a-45c2-a2a8-d905f15c672a admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:18.541548 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc9d22-4f5a-45c2-a2a8-d905f15c672a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:18.544326 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc9d22-4f5a-45c2-a2a8-d905f15c672a admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:18.553595 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:18.578110 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1369/2726] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:42:18 2022] POST /identity/v3/users => generated 285 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:42:18.582760 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1b130-d79e-4906-a768-050ba249350c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:18.595045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1b130-d79e-4906-a768-050ba249350c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:18.602647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bb274-9def-4dc2-974a-2b8001698265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:18.603945 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bb274-9def-4dc2-974a-2b8001698265 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:18.604228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bb274-9def-4dc2-974a-2b8001698265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:18.604504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bb274-9def-4dc2-974a-2b8001698265 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:18.606404 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bb274-9def-4dc2-974a-2b8001698265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:18.609453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bb274-9def-4dc2-974a-2b8001698265 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:18.622866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1358/2727] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:42:18.627159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc9d22-4f5a-45c2-a2a8-d905f15c672a admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:18.644965 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc9d22-4f5a-45c2-a2a8-d905f15c672a admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:18.651144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8417e-afb6-4ec2-962d-d0a732feac4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:18.651874 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8417e-afb6-4ec2-962d-d0a732feac4c admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:18.652114 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8417e-afb6-4ec2-962d-d0a732feac4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:18.652399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8417e-afb6-4ec2-962d-d0a732feac4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:18.653138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b8417e-afb6-4ec2-962d-d0a732feac4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:18.654809 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b8417e-afb6-4ec2-962d-d0a732feac4c admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:18.670437 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1370/2728] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 20:42:18.674714 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bb274-9def-4dc2-974a-2b8001698265 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:18.685952 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bb274-9def-4dc2-974a-2b8001698265 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:18.699664 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962a6e4-1dc6-499e-8972-5a48f4579b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:18.700832 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962a6e4-1dc6-499e-8972-5a48f4579b8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:18.701169 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962a6e4-1dc6-499e-8972-5a48f4579b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:18.701371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962a6e4-1dc6-499e-8972-5a48f4579b8c admin admin] PATH_INFO: `/v3/projects/76d3260d5fbf476299b627a719efa4e9/users/5b47da2c85ba42fd9848a0b7ba17b362/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:18.707252 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:18.727522 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2962a6e4-1dc6-499e-8972-5a48f4579b8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d3260d5fbf476299b627a719efa4e9, user_id=5b47da2c85ba42fd9848a0b7ba17b362, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:18.732053 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2962a6e4-1dc6-499e-8972-5a48f4579b8c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:18.747178 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1359/2729] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:42:18 2022] PUT /identity/v3/projects/76d3260d5fbf476299b627a719efa4e9/users/5b47da2c85ba42fd9848a0b7ba17b362/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:18.757317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cbc184-7a4b-4970-8d5b-05121b777cf2 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:18.757632 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cbc184-7a4b-4970-8d5b-05121b777cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:18.758010 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cbc184-7a4b-4970-8d5b-05121b777cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:18.792287 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-63cbc184-7a4b-4970-8d5b-05121b777cf2 None None] MFA Rules not processed for user `5b47da2c85ba42fd9848a0b7ba17b362`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:18.793853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-63cbc184-7a4b-4970-8d5b-05121b777cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:18.846771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee703a3-e3ae-4de1-a341-c1d8b64c5456 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:18.847323 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee703a3-e3ae-4de1-a341-c1d8b64c5456 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:18.847871 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee703a3-e3ae-4de1-a341-c1d8b64c5456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:18.868328 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-bee703a3-e3ae-4de1-a341-c1d8b64c5456 None None] MFA Rules not processed for user `c0d43152eeb9485e9ff16f38aa63ac22`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:18.869737 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-bee703a3-e3ae-4de1-a341-c1d8b64c5456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:18.892423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1371/2730] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:18.897707 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cbc184-7a4b-4970-8d5b-05121b777cf2 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:18.950864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1360/2731] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:18.990052 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cbc184-7a4b-4970-8d5b-05121b777cf2 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:19.004955 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8f889-cd95-4806-8b41-141746129307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:19.005956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8f889-cd95-4806-8b41-141746129307 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:19.006390 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8f889-cd95-4806-8b41-141746129307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:19.007080 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8f889-cd95-4806-8b41-141746129307 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:19.008168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8f889-cd95-4806-8b41-141746129307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:19.010408 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8f889-cd95-4806-8b41-141746129307 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:19.074430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1372/2732] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:42:18 2022] POST /identity/v3/projects => generated 367 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:42:19.078820 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee703a3-e3ae-4de1-a341-c1d8b64c5456 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:19.100576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee703a3-e3ae-4de1-a341-c1d8b64c5456 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:19.296044 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807dbdf-072f-4379-b16a-1cf918b39cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:19.296650 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807dbdf-072f-4379-b16a-1cf918b39cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:19.297025 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807dbdf-072f-4379-b16a-1cf918b39cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:19.297730 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807dbdf-072f-4379-b16a-1cf918b39cc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:19.298651 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807dbdf-072f-4379-b16a-1cf918b39cc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:19.302295 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807dbdf-072f-4379-b16a-1cf918b39cc5 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:19.334988 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1361/2733] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:42:19 2022] POST /identity/v3/users => generated 290 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:42:19.339217 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8f889-cd95-4806-8b41-141746129307 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:19.357329 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8f889-cd95-4806-8b41-141746129307 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:19.375301 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120daa72-25ed-41a5-bcc9-7980b83792c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:19.375991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120daa72-25ed-41a5-bcc9-7980b83792c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:19.376287 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120daa72-25ed-41a5-bcc9-7980b83792c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:19.376567 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120daa72-25ed-41a5-bcc9-7980b83792c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:19.377316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120daa72-25ed-41a5-bcc9-7980b83792c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:19.379221 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120daa72-25ed-41a5-bcc9-7980b83792c7 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:19.393410 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1373/2734] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:42:19.398621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807dbdf-072f-4379-b16a-1cf918b39cc5 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:19.413672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807dbdf-072f-4379-b16a-1cf918b39cc5 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:19.426411 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235e391-60ed-4944-80aa-8f9761a8ae06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:19.427370 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235e391-60ed-4944-80aa-8f9761a8ae06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:19.427652 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235e391-60ed-4944-80aa-8f9761a8ae06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:19.427994 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235e391-60ed-4944-80aa-8f9761a8ae06 admin admin] PATH_INFO: `/v3/projects/74389b77499d4591babf138807279715/users/17ff176010e14966b419d71a7b8daaf5/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:19.459482 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a235e391-60ed-4944-80aa-8f9761a8ae06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74389b77499d4591babf138807279715, user_id=17ff176010e14966b419d71a7b8daaf5, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:19.463726 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a235e391-60ed-4944-80aa-8f9761a8ae06 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:19.487371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1362/2735] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:42:19 2022] PUT /identity/v3/projects/74389b77499d4591babf138807279715/users/17ff176010e14966b419d71a7b8daaf5/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:19.489284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120daa72-25ed-41a5-bcc9-7980b83792c7 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:19.502630 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120daa72-25ed-41a5-bcc9-7980b83792c7 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:19.573953 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:19.596963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51f270-3e16-4a33-a869-75b6d307f335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:19.597758 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51f270-3e16-4a33-a869-75b6d307f335 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:19.598118 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51f270-3e16-4a33-a869-75b6d307f335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:19.598470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51f270-3e16-4a33-a869-75b6d307f335 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:19.599669 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51f270-3e16-4a33-a869-75b6d307f335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:19.601415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51f270-3e16-4a33-a869-75b6d307f335 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:19.615271 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1374/2736] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:42:19.619630 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235e391-60ed-4944-80aa-8f9761a8ae06 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:19.633100 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235e391-60ed-4944-80aa-8f9761a8ae06 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:19.639210 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ddba0-6887-4986-ae92-6f9b315ef8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:19.640590 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ddba0-6887-4986-ae92-6f9b315ef8de admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:19.641719 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ddba0-6887-4986-ae92-6f9b315ef8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:19.642110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ddba0-6887-4986-ae92-6f9b315ef8de admin admin] PATH_INFO: `/v3/projects/74389b77499d4591babf138807279715/users/17ff176010e14966b419d71a7b8daaf5/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:19.648337 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ddba0-6887-4986-ae92-6f9b315ef8de admin admin] RBAC: Authorizing `identity:create_grant(project_id=74389b77499d4591babf138807279715, user_id=17ff176010e14966b419d71a7b8daaf5, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:19.655280 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ddba0-6887-4986-ae92-6f9b315ef8de admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:19.703742 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1363/2737] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:42:19 2022] PUT /identity/v3/projects/74389b77499d4591babf138807279715/users/17ff176010e14966b419d71a7b8daaf5/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:19.715677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3c743-49cc-47ca-89a1-f093c911081c None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:19.715972 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3c743-49cc-47ca-89a1-f093c911081c None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:19.719323 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3c743-49cc-47ca-89a1-f093c911081c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:19.795385 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-f8c3c743-49cc-47ca-89a1-f093c911081c None None] MFA Rules not processed for user `17ff176010e14966b419d71a7b8daaf5`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:19.796555 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f8c3c743-49cc-47ca-89a1-f093c911081c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:19.942623 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1375/2738] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:19 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:19.951771 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ddba0-6887-4986-ae92-6f9b315ef8de admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:19.954609 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-547ddba0-6887-4986-ae92-6f9b315ef8de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:20.037483 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:20.050755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ddba0-6887-4986-ae92-6f9b315ef8de admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:20.056109 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398f2de-d4ec-4318-8c99-386452415fce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:20.056821 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398f2de-d4ec-4318-8c99-386452415fce service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:20.057110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398f2de-d4ec-4318-8c99-386452415fce service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:20.057386 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398f2de-d4ec-4318-8c99-386452415fce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:20.079584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398f2de-d4ec-4318-8c99-386452415fce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:20.081202 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398f2de-d4ec-4318-8c99-386452415fce service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:20.093848 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1364/2739] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:42:19 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:42:20.139926 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3c743-49cc-47ca-89a1-f093c911081c None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:20.141400 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-f8c3c743-49cc-47ca-89a1-f093c911081c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:20.155210 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3c743-49cc-47ca-89a1-f093c911081c None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:20.159209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7b296-d270-470e-96da-a23dc30c7485 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:20.159633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7b296-d270-470e-96da-a23dc30c7485 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:20.159919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7b296-d270-470e-96da-a23dc30c7485 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:20.160227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7b296-d270-470e-96da-a23dc30c7485 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:20.224962 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7b296-d270-470e-96da-a23dc30c7485 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:20.228053 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7b296-d270-470e-96da-a23dc30c7485 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:20.241251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1376/2740] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:42:20 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:42:23.679348 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398f2de-d4ec-4318-8c99-386452415fce service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:23.744798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398f2de-d4ec-4318-8c99-386452415fce service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:23.749685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a23ea-ea52-405b-937a-808fffdc1760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:23.750625 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a23ea-ea52-405b-937a-808fffdc1760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:23.750910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a23ea-ea52-405b-937a-808fffdc1760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:23.751221 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a23ea-ea52-405b-937a-808fffdc1760 admin admin] PATH_INFO: `/v3/users/5b47da2c85ba42fd9848a0b7ba17b362` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:23.752543 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a23ea-ea52-405b-937a-808fffdc1760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b47da2c85ba42fd9848a0b7ba17b362)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:23.755654 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a23ea-ea52-405b-937a-808fffdc1760 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:23.807283 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9c6a23ea-ea52-405b-937a-808fffdc1760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b47da2c85ba42fd9848a0b7ba17b362'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:23.814855 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9c6a23ea-ea52-405b-937a-808fffdc1760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b47da2c85ba42fd9848a0b7ba17b362'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:23.815487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9c6a23ea-ea52-405b-937a-808fffdc1760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b47da2c85ba42fd9848a0b7ba17b362'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:23.816085 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9c6a23ea-ea52-405b-937a-808fffdc1760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b47da2c85ba42fd9848a0b7ba17b362'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:23.828222 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-9c6a23ea-ea52-405b-937a-808fffdc1760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b47da2c85ba42fd9848a0b7ba17b362'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:23.842617 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1365/2741] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:42:23 2022] DELETE /identity/v3/users/5b47da2c85ba42fd9848a0b7ba17b362 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:42:23.847198 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c03358-645e-4fb0-8316-2d5fb315bc25 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:23.847689 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c03358-645e-4fb0-8316-2d5fb315bc25 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:23.847791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c03358-645e-4fb0-8316-2d5fb315bc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:23.867527 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-b6c03358-645e-4fb0-8316-2d5fb315bc25 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:23.868773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b6c03358-645e-4fb0-8316-2d5fb315bc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:23.929451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1377/2742] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:23.939974 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a23ea-ea52-405b-937a-808fffdc1760 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:23.941738 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9c6a23ea-ea52-405b-937a-808fffdc1760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:24.001380 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a23ea-ea52-405b-937a-808fffdc1760 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:24.005727 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9313f-155d-43e0-a614-291068b8a9cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:24.006453 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9313f-155d-43e0-a614-291068b8a9cb service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:24.006808 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9313f-155d-43e0-a614-291068b8a9cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:24.007294 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9313f-155d-43e0-a614-291068b8a9cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:24.019615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9313f-155d-43e0-a614-291068b8a9cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:24.021171 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9313f-155d-43e0-a614-291068b8a9cb service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:24.032541 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1366/2743] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:42:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:42:24.323427 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c03358-645e-4fb0-8316-2d5fb315bc25 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:24.324877 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b6c03358-645e-4fb0-8316-2d5fb315bc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:24.332093 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:24.337480 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c03358-645e-4fb0-8316-2d5fb315bc25 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:24.344071 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49b763-8a3a-4ec4-bba3-01ae18503ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:24.344943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49b763-8a3a-4ec4-bba3-01ae18503ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:24.345230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49b763-8a3a-4ec4-bba3-01ae18503ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:24.345509 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49b763-8a3a-4ec4-bba3-01ae18503ad4 admin admin] PATH_INFO: `/v3/projects/76d3260d5fbf476299b627a719efa4e9` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:24.346705 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49b763-8a3a-4ec4-bba3-01ae18503ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d3260d5fbf476299b627a719efa4e9)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:24.348645 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49b763-8a3a-4ec4-bba3-01ae18503ad4 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:24.401130 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3b49b763-8a3a-4ec4-bba3-01ae18503ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d3260d5fbf476299b627a719efa4e9', 'request_id': 'req-3b49b763-8a3a-4ec4-bba3-01ae18503ad4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:24.409771 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3b49b763-8a3a-4ec4-bba3-01ae18503ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d3260d5fbf476299b627a719efa4e9', 'request_id': 'req-3b49b763-8a3a-4ec4-bba3-01ae18503ad4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:24.416623 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3b49b763-8a3a-4ec4-bba3-01ae18503ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d3260d5fbf476299b627a719efa4e9', 'request_id': 'req-3b49b763-8a3a-4ec4-bba3-01ae18503ad4'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:24.426770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3b49b763-8a3a-4ec4-bba3-01ae18503ad4 admin admin] The token cache is being invalidate because project 76d3260d5fbf476299b627a719efa4e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:42:24.427095 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3b49b763-8a3a-4ec4-bba3-01ae18503ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:24.427727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3b49b763-8a3a-4ec4-bba3-01ae18503ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:24.429390 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1378/2744] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:42:24 2022] DELETE /identity/v3/projects/76d3260d5fbf476299b627a719efa4e9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:24.433798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9313f-155d-43e0-a614-291068b8a9cb service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:24.443837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9313f-155d-43e0-a614-291068b8a9cb service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:24.448167 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:24.506379 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cdfbe-86f3-44cf-a01a-7556b950f9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:24.507290 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cdfbe-86f3-44cf-a01a-7556b950f9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:24.507577 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cdfbe-86f3-44cf-a01a-7556b950f9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:24.507893 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cdfbe-86f3-44cf-a01a-7556b950f9fd admin admin] PATH_INFO: `/v3/users/17ff176010e14966b419d71a7b8daaf5` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:24.509162 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5cdfbe-86f3-44cf-a01a-7556b950f9fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=17ff176010e14966b419d71a7b8daaf5)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:24.511255 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5cdfbe-86f3-44cf-a01a-7556b950f9fd admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:24.564520 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bf5cdfbe-86f3-44cf-a01a-7556b950f9fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17ff176010e14966b419d71a7b8daaf5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:24.572170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bf5cdfbe-86f3-44cf-a01a-7556b950f9fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17ff176010e14966b419d71a7b8daaf5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:24.573034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bf5cdfbe-86f3-44cf-a01a-7556b950f9fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17ff176010e14966b419d71a7b8daaf5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:24.573807 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bf5cdfbe-86f3-44cf-a01a-7556b950f9fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17ff176010e14966b419d71a7b8daaf5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:24.585851 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bf5cdfbe-86f3-44cf-a01a-7556b950f9fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17ff176010e14966b419d71a7b8daaf5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:24.599063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1367/2745] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:42:24 2022] DELETE /identity/v3/users/17ff176010e14966b419d71a7b8daaf5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:24.883881 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49b763-8a3a-4ec4-bba3-01ae18503ad4 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:24.886743 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3b49b763-8a3a-4ec4-bba3-01ae18503ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:24.904094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49b763-8a3a-4ec4-bba3-01ae18503ad4 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:24.961720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f261b3-070c-495e-93b8-567425b6b50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:24.962701 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f261b3-070c-495e-93b8-567425b6b50e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:24.963006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f261b3-070c-495e-93b8-567425b6b50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:24.963287 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f261b3-070c-495e-93b8-567425b6b50e admin admin] PATH_INFO: `/v3/projects/74389b77499d4591babf138807279715` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:24.964597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f261b3-070c-495e-93b8-567425b6b50e admin admin] RBAC: Authorizing `identity:delete_project(project_id=74389b77499d4591babf138807279715)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:24.966772 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f261b3-070c-495e-93b8-567425b6b50e admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:25.015298 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-32f261b3-070c-495e-93b8-567425b6b50e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74389b77499d4591babf138807279715', 'request_id': 'req-32f261b3-070c-495e-93b8-567425b6b50e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:25.022701 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-32f261b3-070c-495e-93b8-567425b6b50e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74389b77499d4591babf138807279715', 'request_id': 'req-32f261b3-070c-495e-93b8-567425b6b50e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:25.029806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-32f261b3-070c-495e-93b8-567425b6b50e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74389b77499d4591babf138807279715', 'request_id': 'req-32f261b3-070c-495e-93b8-567425b6b50e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:25.042306 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-32f261b3-070c-495e-93b8-567425b6b50e admin admin] The token cache is being invalidate because project 74389b77499d4591babf138807279715 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:42:25.042606 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-32f261b3-070c-495e-93b8-567425b6b50e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:25.043272 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-32f261b3-070c-495e-93b8-567425b6b50e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:42:25.044820 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1379/2746] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:42:24 2022] DELETE /identity/v3/projects/74389b77499d4591babf138807279715 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:25.051667 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff38e2-d089-405b-85bb-17733c207538 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:25.052144 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff38e2-d089-405b-85bb-17733c207538 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:25.052262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff38e2-d089-405b-85bb-17733c207538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:25.072512 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8bff38e2-d089-405b-85bb-17733c207538 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:25.073711 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8bff38e2-d089-405b-85bb-17733c207538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:25.145168 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1368/2747] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:25.153121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50dead4-039e-47ab-bbf1-8f2a34f854b0 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:25.153406 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50dead4-039e-47ab-bbf1-8f2a34f854b0 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:25.153679 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50dead4-039e-47ab-bbf1-8f2a34f854b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:25.173080 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-a50dead4-039e-47ab-bbf1-8f2a34f854b0 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:25.174561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-a50dead4-039e-47ab-bbf1-8f2a34f854b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:25.187308 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1380/2748] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:25.188642 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff38e2-d089-405b-85bb-17733c207538 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:25.200391 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff38e2-d089-405b-85bb-17733c207538 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:25.205330 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a989c7d7-50ba-4cc2-90f5-6e1c76d96f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:25.205996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989c7d7-50ba-4cc2-90f5-6e1c76d96f6b admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:25.206275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989c7d7-50ba-4cc2-90f5-6e1c76d96f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:25.206549 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989c7d7-50ba-4cc2-90f5-6e1c76d96f6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:25.207316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989c7d7-50ba-4cc2-90f5-6e1c76d96f6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:42:25.207615 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989c7d7-50ba-4cc2-90f5-6e1c76d96f6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:25.209225 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989c7d7-50ba-4cc2-90f5-6e1c76d96f6b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:25.235965 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1369/2749] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:42:25.241227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50dead4-039e-47ab-bbf1-8f2a34f854b0 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:25.256309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50dead4-039e-47ab-bbf1-8f2a34f854b0 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:25.264270 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab4c5b-acf9-4071-9b21-bd19f4804a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:25.265293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab4c5b-acf9-4071-9b21-bd19f4804a03 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:25.265798 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab4c5b-acf9-4071-9b21-bd19f4804a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:25.266175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab4c5b-acf9-4071-9b21-bd19f4804a03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:25.267376 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab4c5b-acf9-4071-9b21-bd19f4804a03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:25.270990 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab4c5b-acf9-4071-9b21-bd19f4804a03 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:25.276655 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:25.310106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1381/2750] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:42:25 2022] POST /identity/v3/projects => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:42:25.315341 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a989c7d7-50ba-4cc2-90f5-6e1c76d96f6b admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:25.327026 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a989c7d7-50ba-4cc2-90f5-6e1c76d96f6b admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:25.345704 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:25.401109 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f983ccf-fb32-4592-a267-c7f9ca6f8d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:25.401733 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f983ccf-fb32-4592-a267-c7f9ca6f8d33 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:25.402069 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f983ccf-fb32-4592-a267-c7f9ca6f8d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:25.402233 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f983ccf-fb32-4592-a267-c7f9ca6f8d33 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:25.402944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f983ccf-fb32-4592-a267-c7f9ca6f8d33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:25.404905 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f983ccf-fb32-4592-a267-c7f9ca6f8d33 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:25.426842 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1370/2751] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:42:25 2022] POST /identity/v3/users => generated 291 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:42:25.432206 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab4c5b-acf9-4071-9b21-bd19f4804a03 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:25.444088 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab4c5b-acf9-4071-9b21-bd19f4804a03 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:25.448887 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005834d-ad99-4e2f-9de5-392c642b5a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:25.449548 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005834d-ad99-4e2f-9de5-392c642b5a6b admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:25.449827 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005834d-ad99-4e2f-9de5-392c642b5a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:25.450106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005834d-ad99-4e2f-9de5-392c642b5a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:25.450842 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005834d-ad99-4e2f-9de5-392c642b5a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:25.452460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005834d-ad99-4e2f-9de5-392c642b5a6b admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:25.464898 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1382/2752] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:42:25.469944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f983ccf-fb32-4592-a267-c7f9ca6f8d33 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:25.480837 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f983ccf-fb32-4592-a267-c7f9ca6f8d33 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:25.486111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17a057-3ac0-4149-a124-2ab109137700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:25.486662 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17a057-3ac0-4149-a124-2ab109137700 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:25.486962 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17a057-3ac0-4149-a124-2ab109137700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:25.487267 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17a057-3ac0-4149-a124-2ab109137700 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:25.488159 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17a057-3ac0-4149-a124-2ab109137700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:25.489657 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17a057-3ac0-4149-a124-2ab109137700 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:25.500198 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1371/2753] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:42:25.504479 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005834d-ad99-4e2f-9de5-392c642b5a6b admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:25.515039 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005834d-ad99-4e2f-9de5-392c642b5a6b admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:25.520959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b0997-2292-4fbf-8bcc-35ef8a269e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:25.521842 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b0997-2292-4fbf-8bcc-35ef8a269e5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:25.522130 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b0997-2292-4fbf-8bcc-35ef8a269e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:25.522410 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b0997-2292-4fbf-8bcc-35ef8a269e5d admin admin] PATH_INFO: `/v3/projects/d2fb85822f104b7a981279226f8f1aa6/users/aa4214b07ad54de69e689254dfc99eb3/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:25.538889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b0997-2292-4fbf-8bcc-35ef8a269e5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2fb85822f104b7a981279226f8f1aa6, user_id=aa4214b07ad54de69e689254dfc99eb3, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:25.542229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b0997-2292-4fbf-8bcc-35ef8a269e5d admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:25.556224 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1383/2754] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:42:25 2022] PUT /identity/v3/projects/d2fb85822f104b7a981279226f8f1aa6/users/aa4214b07ad54de69e689254dfc99eb3/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:42:25.561779 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a91a65-0e23-4f71-8d3e-b0e724a04d7a None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:25.562170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a91a65-0e23-4f71-8d3e-b0e724a04d7a None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:25.562518 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a91a65-0e23-4f71-8d3e-b0e724a04d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:25.580450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-37a91a65-0e23-4f71-8d3e-b0e724a04d7a None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:25.581796 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-37a91a65-0e23-4f71-8d3e-b0e724a04d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:25.648683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1372/2755] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:25.658886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b0997-2292-4fbf-8bcc-35ef8a269e5d admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:25.660832 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-061b0997-2292-4fbf-8bcc-35ef8a269e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:25.720657 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b0997-2292-4fbf-8bcc-35ef8a269e5d admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:25.724316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1b66d-f871-40a8-80d9-5fced7f2265e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:25.725184 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1b66d-f871-40a8-80d9-5fced7f2265e service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:25.725625 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1b66d-f871-40a8-80d9-5fced7f2265e service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:25.725988 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1b66d-f871-40a8-80d9-5fced7f2265e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:25.736905 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1b66d-f871-40a8-80d9-5fced7f2265e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:25.738261 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1b66d-f871-40a8-80d9-5fced7f2265e service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:25.748123 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1384/2756] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:42:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:42:26.413711 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f66ab-ee92-42ed-ac33-d9dc80b43230 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:26.414444 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f66ab-ee92-42ed-ac33-d9dc80b43230 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:26.414844 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f66ab-ee92-42ed-ac33-d9dc80b43230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:26.468556 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-876f66ab-ee92-42ed-ac33-d9dc80b43230 None None] MFA Rules not processed for user `aa4214b07ad54de69e689254dfc99eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:26.470272 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-876f66ab-ee92-42ed-ac33-d9dc80b43230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:26.565111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1373/2757] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:26.579128 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1b66d-f871-40a8-80d9-5fced7f2265e service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:26.581125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-08d1b66d-f871-40a8-80d9-5fced7f2265e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:26.654672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1b66d-f871-40a8-80d9-5fced7f2265e service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:26.658525 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9f7c4-4257-41b0-b961-7b76c70484b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:26.659225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9f7c4-4257-41b0-b961-7b76c70484b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:26.659507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9f7c4-4257-41b0-b961-7b76c70484b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:26.659786 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9f7c4-4257-41b0-b961-7b76c70484b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:26.671054 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9f7c4-4257-41b0-b961-7b76c70484b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:26.672694 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9f7c4-4257-41b0-b961-7b76c70484b7 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:26.684152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1385/2758] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:42:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:42:28.124494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba10d0d-6ef3-40a9-a5b3-80049ef0050c None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:28.125061 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba10d0d-6ef3-40a9-a5b3-80049ef0050c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:28.125061 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba10d0d-6ef3-40a9-a5b3-80049ef0050c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:28.146449 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-7ba10d0d-6ef3-40a9-a5b3-80049ef0050c None None] MFA Rules not processed for user `aa4214b07ad54de69e689254dfc99eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:42:28.147681 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7ba10d0d-6ef3-40a9-a5b3-80049ef0050c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:28.155306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1374/2759] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:42:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:42:28.395779 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9f7c4-4257-41b0-b961-7b76c70484b7 service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:28.406314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9f7c4-4257-41b0-b961-7b76c70484b7 service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:28.413023 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3f5fa-8760-4733-95e8-9c23ea84e432 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:28.413677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3f5fa-8760-4733-95e8-9c23ea84e432 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:28.413959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3f5fa-8760-4733-95e8-9c23ea84e432 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:28.414234 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3f5fa-8760-4733-95e8-9c23ea84e432 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:28.416101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ecf3f5fa-8760-4733-95e8-9c23ea84e432 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:28.426279 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3f5fa-8760-4733-95e8-9c23ea84e432 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:28.427725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3f5fa-8760-4733-95e8-9c23ea84e432 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:28.437063 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1386/2760] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:42:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:42:38.140296 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:38.433157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=78,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:42:57.789974 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba10d0d-6ef3-40a9-a5b3-80049ef0050c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:42:57.793749 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7ba10d0d-6ef3-40a9-a5b3-80049ef0050c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:57.806792 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba10d0d-6ef3-40a9-a5b3-80049ef0050c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:42:57.810383 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31236280-78b1-4a3e-b95e-0b9c7b167bf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:42:57.811049 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31236280-78b1-4a3e-b95e-0b9c7b167bf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:42:57.811332 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31236280-78b1-4a3e-b95e-0b9c7b167bf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:42:57.811606 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31236280-78b1-4a3e-b95e-0b9c7b167bf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:42:57.813374 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-31236280-78b1-4a3e-b95e-0b9c7b167bf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:42:57.877761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31236280-78b1-4a3e-b95e-0b9c7b167bf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:42:57.879226 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31236280-78b1-4a3e-b95e-0b9c7b167bf3 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:42:57.889753 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1375/2761] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:42:57 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:43:07.885034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=30 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:43:28.549082 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3f5fa-8760-4733-95e8-9c23ea84e432 service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:43:28.550496 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ecf3f5fa-8760-4733-95e8-9c23ea84e432 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:43:28.646921 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3f5fa-8760-4733-95e8-9c23ea84e432 service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:43:28.652802 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5cd11-6205-4577-9fe0-7b73b5c74ee1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afad68819504c2b864851c6989ae25a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:43:28.653890 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5cd11-6205-4577-9fe0-7b73b5c74ee1 service placement] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:43:28.654383 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5cd11-6205-4577-9fe0-7b73b5c74ee1 service placement] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:43:28.654676 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5cd11-6205-4577-9fe0-7b73b5c74ee1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:43:28.657100 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-5aa5cd11-6205-4577-9fe0-7b73b5c74ee1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:43:28.670773 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5cd11-6205-4577-9fe0-7b73b5c74ee1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:43:28.672413 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5cd11-6205-4577-9fe0-7b73b5c74ee1 service placement] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:43:28.683642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1387/2762] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:43:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:43:38.678417 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=41 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:43:39.674230 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c447e4-c02d-4bee-b89d-d370ab50145c None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:43:39.674810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c447e4-c02d-4bee-b89d-d370ab50145c None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:43:39.674810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c447e4-c02d-4bee-b89d-d370ab50145c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:43:39.692560 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-19c447e4-c02d-4bee-b89d-d370ab50145c None None] MFA Rules not processed for user `aa4214b07ad54de69e689254dfc99eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:43:39.693779 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-19c447e4-c02d-4bee-b89d-d370ab50145c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:43:39.701234 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1376/2763] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:43:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:43:43.881181 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719ffbf-004c-4a5b-a8a9-09cd6cec3d54 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:43:43.881680 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719ffbf-004c-4a5b-a8a9-09cd6cec3d54 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:43:43.882203 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719ffbf-004c-4a5b-a8a9-09cd6cec3d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:43:43.900898 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-e719ffbf-004c-4a5b-a8a9-09cd6cec3d54 None None] MFA Rules not processed for user `813a2d3222f54b18a4322174a409183e`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:43:43.902243 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e719ffbf-004c-4a5b-a8a9-09cd6cec3d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:43:43.963620 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1388/2764] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:43:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:43:49.687113 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:43:53.956593 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=25 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:44:38.931190 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c447e4-c02d-4bee-b89d-d370ab50145c None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:44:38.942579 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c447e4-c02d-4bee-b89d-d370ab50145c None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:44:38.946921 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d669d-101e-4c49-82fb-fd0a6bd2ce07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:44:38.947604 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d669d-101e-4c49-82fb-fd0a6bd2ce07 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:44:38.947887 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d669d-101e-4c49-82fb-fd0a6bd2ce07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:44:38.948162 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d669d-101e-4c49-82fb-fd0a6bd2ce07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:44:38.949995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9e9d669d-101e-4c49-82fb-fd0a6bd2ce07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:44:38.963053 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d669d-101e-4c49-82fb-fd0a6bd2ce07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:44:38.964399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d669d-101e-4c49-82fb-fd0a6bd2ce07 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:44:38.975108 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1377/2765] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:44:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:44:41.689609 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719ffbf-004c-4a5b-a8a9-09cd6cec3d54 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:44:41.701171 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719ffbf-004c-4a5b-a8a9-09cd6cec3d54 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:44:41.705823 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2819a-6811-44dc-a6dc-a61ef3e79508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:44:41.706709 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2819a-6811-44dc-a6dc-a61ef3e79508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:44:41.707559 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2819a-6811-44dc-a6dc-a61ef3e79508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:44:41.707845 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2819a-6811-44dc-a6dc-a61ef3e79508 admin admin] PATH_INFO: `/v3/users/aa4214b07ad54de69e689254dfc99eb3` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:44:41.709073 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2819a-6811-44dc-a6dc-a61ef3e79508 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa4214b07ad54de69e689254dfc99eb3)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:44:41.711085 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2819a-6811-44dc-a6dc-a61ef3e79508 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:44:41.761812 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-fad2819a-6811-44dc-a6dc-a61ef3e79508 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa4214b07ad54de69e689254dfc99eb3'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:41.770577 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-fad2819a-6811-44dc-a6dc-a61ef3e79508 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa4214b07ad54de69e689254dfc99eb3'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:41.777741 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-fad2819a-6811-44dc-a6dc-a61ef3e79508 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa4214b07ad54de69e689254dfc99eb3'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:41.778386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-fad2819a-6811-44dc-a6dc-a61ef3e79508 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa4214b07ad54de69e689254dfc99eb3'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:41.779027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-fad2819a-6811-44dc-a6dc-a61ef3e79508 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa4214b07ad54de69e689254dfc99eb3'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:41.799242 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1389/2766] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:44:41 2022] DELETE /identity/v3/users/aa4214b07ad54de69e689254dfc99eb3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:44:41.989904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d669d-101e-4c49-82fb-fd0a6bd2ce07 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:44:42.011181 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d669d-101e-4c49-82fb-fd0a6bd2ce07 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:44:42.128371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed767ad-4c7a-47ad-b81b-ff2a114a80da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:44:42.131595 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed767ad-4c7a-47ad-b81b-ff2a114a80da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:44:42.132063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed767ad-4c7a-47ad-b81b-ff2a114a80da admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:44:42.133180 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed767ad-4c7a-47ad-b81b-ff2a114a80da admin admin] PATH_INFO: `/v3/projects/d2fb85822f104b7a981279226f8f1aa6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:44:42.135042 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed767ad-4c7a-47ad-b81b-ff2a114a80da admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2fb85822f104b7a981279226f8f1aa6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:44:42.138198 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed767ad-4c7a-47ad-b81b-ff2a114a80da admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:44:42.213202 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1ed767ad-4c7a-47ad-b81b-ff2a114a80da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2fb85822f104b7a981279226f8f1aa6', 'request_id': 'req-1ed767ad-4c7a-47ad-b81b-ff2a114a80da'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:42.222145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1ed767ad-4c7a-47ad-b81b-ff2a114a80da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2fb85822f104b7a981279226f8f1aa6', 'request_id': 'req-1ed767ad-4c7a-47ad-b81b-ff2a114a80da'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:42.233593 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1ed767ad-4c7a-47ad-b81b-ff2a114a80da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2fb85822f104b7a981279226f8f1aa6', 'request_id': 'req-1ed767ad-4c7a-47ad-b81b-ff2a114a80da'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:42.251764 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1ed767ad-4c7a-47ad-b81b-ff2a114a80da admin admin] The token cache is being invalidate because project d2fb85822f104b7a981279226f8f1aa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:44:42.252056 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1ed767ad-4c7a-47ad-b81b-ff2a114a80da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:42.253174 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-1ed767ad-4c7a-47ad-b81b-ff2a114a80da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:42.259159 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1378/2767] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:44:41 2022] DELETE /identity/v3/projects/d2fb85822f104b7a981279226f8f1aa6 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:44:45.708202 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2819a-6811-44dc-a6dc-a61ef3e79508 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:44:45.710753 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-fad2819a-6811-44dc-a6dc-a61ef3e79508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:44:45.727952 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2819a-6811-44dc-a6dc-a61ef3e79508 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:44:45.789511 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe882f-a106-4a12-97d9-a94621814424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:44:45.790403 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe882f-a106-4a12-97d9-a94621814424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:44:45.790688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe882f-a106-4a12-97d9-a94621814424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:44:45.790991 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe882f-a106-4a12-97d9-a94621814424 admin admin] PATH_INFO: `/v3/users/6502e3ffa3b84333a00ea5f712767d61` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:44:45.792233 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe882f-a106-4a12-97d9-a94621814424 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6502e3ffa3b84333a00ea5f712767d61)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:44:45.794159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe882f-a106-4a12-97d9-a94621814424 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:44:45.841593 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3abe882f-a106-4a12-97d9-a94621814424 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6502e3ffa3b84333a00ea5f712767d61'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:45.849650 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3abe882f-a106-4a12-97d9-a94621814424 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6502e3ffa3b84333a00ea5f712767d61'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:45.859885 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3abe882f-a106-4a12-97d9-a94621814424 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6502e3ffa3b84333a00ea5f712767d61'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:45.860611 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3abe882f-a106-4a12-97d9-a94621814424 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6502e3ffa3b84333a00ea5f712767d61'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:45.861276 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-3abe882f-a106-4a12-97d9-a94621814424 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6502e3ffa3b84333a00ea5f712767d61'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:45.875676 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:44:45.887987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1390/2768] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:44:45 2022] DELETE /identity/v3/users/6502e3ffa3b84333a00ea5f712767d61 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:44:46.071306 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed767ad-4c7a-47ad-b81b-ff2a114a80da admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:44:46.072731 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1ed767ad-4c7a-47ad-b81b-ff2a114a80da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:44:46.087031 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed767ad-4c7a-47ad-b81b-ff2a114a80da admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:44:46.143555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4c5e5-20ae-4b56-9897-98133f096252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:44:46.144675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4c5e5-20ae-4b56-9897-98133f096252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:44:46.144957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4c5e5-20ae-4b56-9897-98133f096252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:44:46.145232 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4c5e5-20ae-4b56-9897-98133f096252 admin admin] PATH_INFO: `/v3/projects/83a29cbfb7484b76a9131a3b81832c62` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:44:46.146516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4c5e5-20ae-4b56-9897-98133f096252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83a29cbfb7484b76a9131a3b81832c62)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:44:46.149086 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4c5e5-20ae-4b56-9897-98133f096252 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:44:46.171164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:44:46.197157 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bcb4c5e5-20ae-4b56-9897-98133f096252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83a29cbfb7484b76a9131a3b81832c62', 'request_id': 'req-bcb4c5e5-20ae-4b56-9897-98133f096252'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:46.203555 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bcb4c5e5-20ae-4b56-9897-98133f096252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83a29cbfb7484b76a9131a3b81832c62', 'request_id': 'req-bcb4c5e5-20ae-4b56-9897-98133f096252'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:46.209729 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bcb4c5e5-20ae-4b56-9897-98133f096252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83a29cbfb7484b76a9131a3b81832c62', 'request_id': 'req-bcb4c5e5-20ae-4b56-9897-98133f096252'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:46.222282 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bcb4c5e5-20ae-4b56-9897-98133f096252 admin admin] The token cache is being invalidate because project 83a29cbfb7484b76a9131a3b81832c62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:44:46.222506 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bcb4c5e5-20ae-4b56-9897-98133f096252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:46.223203 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-bcb4c5e5-20ae-4b56-9897-98133f096252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:46.226246 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1379/2769] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:44:46 2022] DELETE /identity/v3/projects/83a29cbfb7484b76a9131a3b81832c62 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:44:46.230209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe882f-a106-4a12-97d9-a94621814424 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:44:46.231938 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3abe882f-a106-4a12-97d9-a94621814424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:44:46.247523 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe882f-a106-4a12-97d9-a94621814424 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:44:46.304687 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975273c-999c-44fd-8a90-7e50dc44d509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:44:46.305596 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975273c-999c-44fd-8a90-7e50dc44d509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:44:46.305879 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975273c-999c-44fd-8a90-7e50dc44d509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:44:46.306156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975273c-999c-44fd-8a90-7e50dc44d509 admin admin] PATH_INFO: `/v3/users/a1f5c428bab24681b4b91bc04f200f7b` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:44:46.307355 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975273c-999c-44fd-8a90-7e50dc44d509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1f5c428bab24681b4b91bc04f200f7b)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:44:46.309273 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975273c-999c-44fd-8a90-7e50dc44d509 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:44:46.354768 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-5975273c-999c-44fd-8a90-7e50dc44d509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1f5c428bab24681b4b91bc04f200f7b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:46.363453 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-5975273c-999c-44fd-8a90-7e50dc44d509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1f5c428bab24681b4b91bc04f200f7b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:46.371233 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-5975273c-999c-44fd-8a90-7e50dc44d509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1f5c428bab24681b4b91bc04f200f7b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:46.371928 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-5975273c-999c-44fd-8a90-7e50dc44d509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1f5c428bab24681b4b91bc04f200f7b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:46.372558 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-5975273c-999c-44fd-8a90-7e50dc44d509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1f5c428bab24681b4b91bc04f200f7b'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:46.389662 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1391/2770] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:44:46 2022] DELETE /identity/v3/users/a1f5c428bab24681b4b91bc04f200f7b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:44:46.570694 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4c5e5-20ae-4b56-9897-98133f096252 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:44:46.572236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-bcb4c5e5-20ae-4b56-9897-98133f096252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:44:46.586372 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4c5e5-20ae-4b56-9897-98133f096252 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:44:46.645881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fa550-5997-44f1-b92d-bcf8f87c686f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:44:46.647319 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fa550-5997-44f1-b92d-bcf8f87c686f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:44:46.647602 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fa550-5997-44f1-b92d-bcf8f87c686f admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:44:46.647876 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fa550-5997-44f1-b92d-bcf8f87c686f admin admin] PATH_INFO: `/v3/projects/8198771fe32f4a978926e04d278220ba` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:44:46.649057 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fa550-5997-44f1-b92d-bcf8f87c686f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8198771fe32f4a978926e04d278220ba)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:44:46.650987 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fa550-5997-44f1-b92d-bcf8f87c686f admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:44:46.697028 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-978fa550-5997-44f1-b92d-bcf8f87c686f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8198771fe32f4a978926e04d278220ba', 'request_id': 'req-978fa550-5997-44f1-b92d-bcf8f87c686f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:46.703096 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-978fa550-5997-44f1-b92d-bcf8f87c686f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8198771fe32f4a978926e04d278220ba', 'request_id': 'req-978fa550-5997-44f1-b92d-bcf8f87c686f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:46.708623 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-978fa550-5997-44f1-b92d-bcf8f87c686f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8198771fe32f4a978926e04d278220ba', 'request_id': 'req-978fa550-5997-44f1-b92d-bcf8f87c686f'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:46.721339 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-978fa550-5997-44f1-b92d-bcf8f87c686f admin admin] The token cache is being invalidate because project 8198771fe32f4a978926e04d278220ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:44:46.721629 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-978fa550-5997-44f1-b92d-bcf8f87c686f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:46.722254 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-978fa550-5997-44f1-b92d-bcf8f87c686f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:44:46.723835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1380/2771] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:44:46 2022] DELETE /identity/v3/projects/8198771fe32f4a978926e04d278220ba => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:44:50.479619 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975273c-999c-44fd-8a90-7e50dc44d509 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:44:50.481405 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-5975273c-999c-44fd-8a90-7e50dc44d509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:44:50.542735 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975273c-999c-44fd-8a90-7e50dc44d509 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:44:50.565690 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:44:50.593906 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992176e-b143-4365-90bd-dcb366c9581c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:44:50.594561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992176e-b143-4365-90bd-dcb366c9581c service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:44:50.594842 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992176e-b143-4365-90bd-dcb366c9581c service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:44:50.595144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992176e-b143-4365-90bd-dcb366c9581c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:44:50.596947 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6992176e-b143-4365-90bd-dcb366c9581c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:44:50.685376 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992176e-b143-4365-90bd-dcb366c9581c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:44:50.686814 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992176e-b143-4365-90bd-dcb366c9581c service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:44:50.707207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1392/2772] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:44:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:44:56.718205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=67,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:00.014818 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fa550-5997-44f1-b92d-bcf8f87c686f admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:00.016262 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-978fa550-5997-44f1-b92d-bcf8f87c686f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:00.029265 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fa550-5997-44f1-b92d-bcf8f87c686f admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:00.032828 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3cecd2-db13-4304-99ed-404bd3f6c99a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:00.033464 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cecd2-db13-4304-99ed-404bd3f6c99a service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:00.033744 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cecd2-db13-4304-99ed-404bd3f6c99a service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:00.034016 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cecd2-db13-4304-99ed-404bd3f6c99a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:00.035747 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4d3cecd2-db13-4304-99ed-404bd3f6c99a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:00.049903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3cecd2-db13-4304-99ed-404bd3f6c99a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:00.051443 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3cecd2-db13-4304-99ed-404bd3f6c99a service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:00.062465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1381/2773] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:45:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:45:00.704727 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=54 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:06.614243 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992176e-b143-4365-90bd-dcb366c9581c service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:06.616314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-6992176e-b143-4365-90bd-dcb366c9581c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:06.627242 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992176e-b143-4365-90bd-dcb366c9581c service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:06.630811 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7f13d-8ee6-440e-a383-fa7f70bec396 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:06.631486 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7f13d-8ee6-440e-a383-fa7f70bec396 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:06.631767 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7f13d-8ee6-440e-a383-fa7f70bec396 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:06.632040 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7f13d-8ee6-440e-a383-fa7f70bec396 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:06.633840 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-28a7f13d-8ee6-440e-a383-fa7f70bec396 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:06.694719 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7f13d-8ee6-440e-a383-fa7f70bec396 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:06.696191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7f13d-8ee6-440e-a383-fa7f70bec396 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:06.705895 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1393/2774] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:45:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:45:10.057546 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=6 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:14.411243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3cecd2-db13-4304-99ed-404bd3f6c99a service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:14.413389 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4d3cecd2-db13-4304-99ed-404bd3f6c99a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:14.428994 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3cecd2-db13-4304-99ed-404bd3f6c99a service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:14.435032 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ce305-2c90-4026-b0cf-cddac431b631 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:14.435891 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ce305-2c90-4026-b0cf-cddac431b631 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:14.436176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ce305-2c90-4026-b0cf-cddac431b631 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:14.436507 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ce305-2c90-4026-b0cf-cddac431b631 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:14.438812 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-1f6ce305-2c90-4026-b0cf-cddac431b631 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:14.455597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ce305-2c90-4026-b0cf-cddac431b631 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:14.457382 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ce305-2c90-4026-b0cf-cddac431b631 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:14.471159 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1382/2775] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:45:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:45:16.701443 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=28 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:20.307573 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7f13d-8ee6-440e-a383-fa7f70bec396 service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:20.309102 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-28a7f13d-8ee6-440e-a383-fa7f70bec396 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:20.369106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7f13d-8ee6-440e-a383-fa7f70bec396 service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:20.372833 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a762e88-bc4c-4e0b-9c74-7563184f6d48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:20.373485 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a762e88-bc4c-4e0b-9c74-7563184f6d48 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:20.373774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a762e88-bc4c-4e0b-9c74-7563184f6d48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:20.374055 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a762e88-bc4c-4e0b-9c74-7563184f6d48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:20.375905 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-0a762e88-bc4c-4e0b-9c74-7563184f6d48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:20.440729 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a762e88-bc4c-4e0b-9c74-7563184f6d48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:20.442162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a762e88-bc4c-4e0b-9c74-7563184f6d48 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:20.453230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1394/2776] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:45:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:45:22.985804 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ce305-2c90-4026-b0cf-cddac431b631 service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:22.997058 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ce305-2c90-4026-b0cf-cddac431b631 service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:23.000769 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079ea8d-7af1-4f0d-95ad-e8b7586e1fb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:23.001471 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079ea8d-7af1-4f0d-95ad-e8b7586e1fb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:23.001945 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079ea8d-7af1-4f0d-95ad-e8b7586e1fb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:23.002399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079ea8d-7af1-4f0d-95ad-e8b7586e1fb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:23.004279 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9079ea8d-7af1-4f0d-95ad-e8b7586e1fb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:23.019718 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9079ea8d-7af1-4f0d-95ad-e8b7586e1fb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:23.021559 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9079ea8d-7af1-4f0d-95ad-e8b7586e1fb2 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:23.032010 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1383/2777] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:45:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:45:23.417233 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a762e88-bc4c-4e0b-9c74-7563184f6d48 service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:23.419993 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-0a762e88-bc4c-4e0b-9c74-7563184f6d48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:23.479087 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079ea8d-7af1-4f0d-95ad-e8b7586e1fb2 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:23.482145 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-9079ea8d-7af1-4f0d-95ad-e8b7586e1fb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:23.510099 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079ea8d-7af1-4f0d-95ad-e8b7586e1fb2 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:23.519884 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03e559-d9b3-44d9-815c-6d30048b6382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:23.520762 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03e559-d9b3-44d9-815c-6d30048b6382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:23.521244 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03e559-d9b3-44d9-815c-6d30048b6382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:23.521688 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03e559-d9b3-44d9-815c-6d30048b6382 admin admin] PATH_INFO: `/v3/users/813a2d3222f54b18a4322174a409183e` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:23.523782 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec03e559-d9b3-44d9-815c-6d30048b6382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=813a2d3222f54b18a4322174a409183e)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:23.525906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec03e559-d9b3-44d9-815c-6d30048b6382 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:23.541097 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a762e88-bc4c-4e0b-9c74-7563184f6d48 service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:23.547295 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061289ea-f995-4f1e-a289-c8aa6633aa54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afad68819504c2b864851c6989ae25a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:23.547956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061289ea-f995-4f1e-a289-c8aa6633aa54 service placement] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:23.548239 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061289ea-f995-4f1e-a289-c8aa6633aa54 service placement] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:23.548519 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061289ea-f995-4f1e-a289-c8aa6633aa54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:23.551244 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-061289ea-f995-4f1e-a289-c8aa6633aa54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:23.566531 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061289ea-f995-4f1e-a289-c8aa6633aa54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:23.568394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061289ea-f995-4f1e-a289-c8aa6633aa54 service placement] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:23.584386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1395/2778] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:45:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:45:23.590683 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ec03e559-d9b3-44d9-815c-6d30048b6382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '813a2d3222f54b18a4322174a409183e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:23.604794 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ec03e559-d9b3-44d9-815c-6d30048b6382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '813a2d3222f54b18a4322174a409183e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:23.605785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ec03e559-d9b3-44d9-815c-6d30048b6382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '813a2d3222f54b18a4322174a409183e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:23.605991 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ec03e559-d9b3-44d9-815c-6d30048b6382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '813a2d3222f54b18a4322174a409183e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:23.621161 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ec03e559-d9b3-44d9-815c-6d30048b6382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '813a2d3222f54b18a4322174a409183e'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:23.635257 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1384/2779] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:45:23 2022] DELETE /identity/v3/users/813a2d3222f54b18a4322174a409183e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:45:23.820919 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061289ea-f995-4f1e-a289-c8aa6633aa54 service placement] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:23.831956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061289ea-f995-4f1e-a289-c8aa6633aa54 service placement] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:23.882515 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:23.917957 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d7292-817f-49a7-81db-007ef0e82b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:23.918870 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d7292-817f-49a7-81db-007ef0e82b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:23.919196 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d7292-817f-49a7-81db-007ef0e82b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:23.919483 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d7292-817f-49a7-81db-007ef0e82b68 admin admin] PATH_INFO: `/v3/projects/01d544a1fa8b4ac08d4588eab1a84ff5` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:23.920880 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d7292-817f-49a7-81db-007ef0e82b68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01d544a1fa8b4ac08d4588eab1a84ff5)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:23.922781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d7292-817f-49a7-81db-007ef0e82b68 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:23.978884 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-5a5d7292-817f-49a7-81db-007ef0e82b68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01d544a1fa8b4ac08d4588eab1a84ff5', 'request_id': 'req-5a5d7292-817f-49a7-81db-007ef0e82b68'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:23.987180 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-5a5d7292-817f-49a7-81db-007ef0e82b68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01d544a1fa8b4ac08d4588eab1a84ff5', 'request_id': 'req-5a5d7292-817f-49a7-81db-007ef0e82b68'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:23.994341 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-5a5d7292-817f-49a7-81db-007ef0e82b68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01d544a1fa8b4ac08d4588eab1a84ff5', 'request_id': 'req-5a5d7292-817f-49a7-81db-007ef0e82b68'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:24.005964 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-5a5d7292-817f-49a7-81db-007ef0e82b68 admin admin] The token cache is being invalidate because project 01d544a1fa8b4ac08d4588eab1a84ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:45:24.006254 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-5a5d7292-817f-49a7-81db-007ef0e82b68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:24.006996 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-5a5d7292-817f-49a7-81db-007ef0e82b68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:24.010978 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1396/2780] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:45:23 2022] DELETE /identity/v3/projects/01d544a1fa8b4ac08d4588eab1a84ff5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:45:24.014314 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c53c1-edea-4881-a1d0-ecf105564f71 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:24.014857 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c53c1-edea-4881-a1d0-ecf105564f71 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:24.014857 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c53c1-edea-4881-a1d0-ecf105564f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:24.034618 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-c11c53c1-edea-4881-a1d0-ecf105564f71 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:45:24.035877 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-c11c53c1-edea-4881-a1d0-ecf105564f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:24.097494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1385/2781] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:45:24.105864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedde25c-d128-44b5-b4a3-85640408dadb None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:24.106155 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedde25c-d128-44b5-b4a3-85640408dadb None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:24.106437 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedde25c-d128-44b5-b4a3-85640408dadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:24.125656 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-dedde25c-d128-44b5-b4a3-85640408dadb None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:45:24.126861 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-dedde25c-d128-44b5-b4a3-85640408dadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:24.135884 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1397/2782] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:45:24.140327 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c53c1-edea-4881-a1d0-ecf105564f71 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:24.150556 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c53c1-edea-4881-a1d0-ecf105564f71 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:24.155803 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ace656-67a3-4a4b-9d0b-585b4e42985a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:24.156478 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ace656-67a3-4a4b-9d0b-585b4e42985a admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:24.156763 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ace656-67a3-4a4b-9d0b-585b4e42985a admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:24.157033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ace656-67a3-4a4b-9d0b-585b4e42985a admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:24.157760 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ace656-67a3-4a4b-9d0b-585b4e42985a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:45:24.158055 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ace656-67a3-4a4b-9d0b-585b4e42985a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:24.159689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ace656-67a3-4a4b-9d0b-585b4e42985a admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:24.172400 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1386/2783] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:45:24.177185 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedde25c-d128-44b5-b4a3-85640408dadb None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:24.187506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedde25c-d128-44b5-b4a3-85640408dadb None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:24.192166 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353f924-9008-43e0-a8e2-9757696672cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:24.192812 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353f924-9008-43e0-a8e2-9757696672cc admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:24.193094 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353f924-9008-43e0-a8e2-9757696672cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:24.193365 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353f924-9008-43e0-a8e2-9757696672cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:24.194061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353f924-9008-43e0-a8e2-9757696672cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:24.196582 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353f924-9008-43e0-a8e2-9757696672cc admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:24.223866 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1398/2784] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:45:24 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:45:24.228198 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ace656-67a3-4a4b-9d0b-585b4e42985a admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:24.238922 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ace656-67a3-4a4b-9d0b-585b4e42985a admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:24.270774 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:24.298915 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445bec5-57f8-4d25-9b86-d4a5a90fb8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:24.299632 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445bec5-57f8-4d25-9b86-d4a5a90fb8f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:24.299909 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445bec5-57f8-4d25-9b86-d4a5a90fb8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:24.300179 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445bec5-57f8-4d25-9b86-d4a5a90fb8f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:24.300872 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445bec5-57f8-4d25-9b86-d4a5a90fb8f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:24.302813 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445bec5-57f8-4d25-9b86-d4a5a90fb8f4 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:24.323176 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1387/2785] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:45:24 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:45:24.327317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353f924-9008-43e0-a8e2-9757696672cc admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:24.337672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353f924-9008-43e0-a8e2-9757696672cc admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:24.342438 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c308249-ac04-4a24-9ca0-71f0a917a6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:24.343128 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c308249-ac04-4a24-9ca0-71f0a917a6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:24.343415 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c308249-ac04-4a24-9ca0-71f0a917a6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:24.343695 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c308249-ac04-4a24-9ca0-71f0a917a6b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:24.344449 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c308249-ac04-4a24-9ca0-71f0a917a6b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:24.346044 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c308249-ac04-4a24-9ca0-71f0a917a6b1 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:24.356633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1399/2786] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:45:24.360433 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445bec5-57f8-4d25-9b86-d4a5a90fb8f4 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:24.371311 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445bec5-57f8-4d25-9b86-d4a5a90fb8f4 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:24.375934 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eecfd3-d8f9-4c75-a338-a379f2919820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:24.376639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eecfd3-d8f9-4c75-a338-a379f2919820 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:24.376916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eecfd3-d8f9-4c75-a338-a379f2919820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:24.377186 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eecfd3-d8f9-4c75-a338-a379f2919820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:24.377936 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eecfd3-d8f9-4c75-a338-a379f2919820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:24.379591 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eecfd3-d8f9-4c75-a338-a379f2919820 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:24.391799 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1388/2787] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:45:24.396471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c308249-ac04-4a24-9ca0-71f0a917a6b1 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:24.406994 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c308249-ac04-4a24-9ca0-71f0a917a6b1 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:24.411979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a13c4-f729-4768-a5e0-89c3452465ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:24.412992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a13c4-f729-4768-a5e0-89c3452465ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:24.413280 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a13c4-f729-4768-a5e0-89c3452465ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:24.414064 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a13c4-f729-4768-a5e0-89c3452465ef admin admin] PATH_INFO: `/v3/projects/147461e41352426493251ac0f6a65a1f/users/f41df8d3c64b45c695cb9420c9c22431/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:24.429672 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a13c4-f729-4768-a5e0-89c3452465ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=147461e41352426493251ac0f6a65a1f, user_id=f41df8d3c64b45c695cb9420c9c22431, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:24.432379 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a13c4-f729-4768-a5e0-89c3452465ef admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:24.445336 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1400/2788] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:45:24 2022] PUT /identity/v3/projects/147461e41352426493251ac0f6a65a1f/users/f41df8d3c64b45c695cb9420c9c22431/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:45:24.455677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bed8b1-517b-4b20-bef4-6bb32d3a8f19 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:24.456251 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bed8b1-517b-4b20-bef4-6bb32d3a8f19 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:24.456666 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bed8b1-517b-4b20-bef4-6bb32d3a8f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:24.483957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a4bed8b1-517b-4b20-bef4-6bb32d3a8f19 None None] MFA Rules not processed for user `f41df8d3c64b45c695cb9420c9c22431`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:45:24.485170 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a4bed8b1-517b-4b20-bef4-6bb32d3a8f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:24.550457 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1389/2789] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:45:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:45:24.555614 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a13c4-f729-4768-a5e0-89c3452465ef admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:24.620881 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a13c4-f729-4768-a5e0-89c3452465ef admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:24.625765 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2744b2-e808-4558-a845-ba52588107e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:24.626438 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2744b2-e808-4558-a845-ba52588107e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:24.626725 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2744b2-e808-4558-a845-ba52588107e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:24.627162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2744b2-e808-4558-a845-ba52588107e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:24.627909 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2744b2-e808-4558-a845-ba52588107e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:24.629821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2744b2-e808-4558-a845-ba52588107e8 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:24.640517 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:24.660462 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1401/2790] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:45:24 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:45:24.661722 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bed8b1-517b-4b20-bef4-6bb32d3a8f19 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:24.673627 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bed8b1-517b-4b20-bef4-6bb32d3a8f19 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:24.730598 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471e56d-f1bf-479b-940f-9e98d2fce314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:24.731302 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471e56d-f1bf-479b-940f-9e98d2fce314 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:24.731588 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471e56d-f1bf-479b-940f-9e98d2fce314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:24.731864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471e56d-f1bf-479b-940f-9e98d2fce314 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:24.732566 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471e56d-f1bf-479b-940f-9e98d2fce314 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:24.734490 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471e56d-f1bf-479b-940f-9e98d2fce314 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:24.755231 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1390/2791] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:45:24 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:45:24.759525 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2744b2-e808-4558-a845-ba52588107e8 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:24.770500 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2744b2-e808-4558-a845-ba52588107e8 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:24.775648 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d15d3-22b5-42a7-885f-19ad5ae556d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:24.776300 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d15d3-22b5-42a7-885f-19ad5ae556d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:24.776581 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d15d3-22b5-42a7-885f-19ad5ae556d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:24.776855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d15d3-22b5-42a7-885f-19ad5ae556d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:24.777576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d15d3-22b5-42a7-885f-19ad5ae556d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:24.779186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d15d3-22b5-42a7-885f-19ad5ae556d2 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:24.790330 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1402/2792] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:45:24.794590 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471e56d-f1bf-479b-940f-9e98d2fce314 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:24.805235 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471e56d-f1bf-479b-940f-9e98d2fce314 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:24.809798 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b215c-5d0f-4838-9bf7-ff5436a7a298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:24.810662 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b215c-5d0f-4838-9bf7-ff5436a7a298 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:24.810961 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b215c-5d0f-4838-9bf7-ff5436a7a298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:24.811251 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b215c-5d0f-4838-9bf7-ff5436a7a298 admin admin] PATH_INFO: `/v3/projects/54413de834e14bab983f236e6c3345d6/users/a5adba51e31c424ea8db196d59e389f2/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:24.826565 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b215c-5d0f-4838-9bf7-ff5436a7a298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54413de834e14bab983f236e6c3345d6, user_id=a5adba51e31c424ea8db196d59e389f2, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:24.829184 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b215c-5d0f-4838-9bf7-ff5436a7a298 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:24.841640 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1391/2793] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:45:24 2022] PUT /identity/v3/projects/54413de834e14bab983f236e6c3345d6/users/a5adba51e31c424ea8db196d59e389f2/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:45:24.845571 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d15d3-22b5-42a7-885f-19ad5ae556d2 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:24.857388 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d15d3-22b5-42a7-885f-19ad5ae556d2 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:24.915578 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071311e-cf4a-415a-b85c-6b4481e043c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:24.916248 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071311e-cf4a-415a-b85c-6b4481e043c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:24.916537 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071311e-cf4a-415a-b85c-6b4481e043c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:24.916816 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071311e-cf4a-415a-b85c-6b4481e043c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:24.917541 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071311e-cf4a-415a-b85c-6b4481e043c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:24.919163 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071311e-cf4a-415a-b85c-6b4481e043c1 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:24.928766 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1403/2794] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:45:24.933584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b215c-5d0f-4838-9bf7-ff5436a7a298 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:24.944022 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b215c-5d0f-4838-9bf7-ff5436a7a298 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:24.949504 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97342bde-ca56-4972-ac0b-885e413a5a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:24.950379 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97342bde-ca56-4972-ac0b-885e413a5a74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:24.950682 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97342bde-ca56-4972-ac0b-885e413a5a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:24.951034 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97342bde-ca56-4972-ac0b-885e413a5a74 admin admin] PATH_INFO: `/v3/projects/54413de834e14bab983f236e6c3345d6/users/a5adba51e31c424ea8db196d59e389f2/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:24.953027 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97342bde-ca56-4972-ac0b-885e413a5a74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54413de834e14bab983f236e6c3345d6, user_id=a5adba51e31c424ea8db196d59e389f2, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:24.955693 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97342bde-ca56-4972-ac0b-885e413a5a74 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:24.958307 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:24.970292 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1392/2795] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:45:24 2022] PUT /identity/v3/projects/54413de834e14bab983f236e6c3345d6/users/a5adba51e31c424ea8db196d59e389f2/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:45:24.980463 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f22061-488f-461d-84f4-35e3c4aed02c None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:24.980755 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f22061-488f-461d-84f4-35e3c4aed02c None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:24.981034 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f22061-488f-461d-84f4-35e3c4aed02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:25.010253 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-80f22061-488f-461d-84f4-35e3c4aed02c None None] MFA Rules not processed for user `a5adba51e31c424ea8db196d59e389f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:45:25.011582 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-80f22061-488f-461d-84f4-35e3c4aed02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:25.086003 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1404/2796] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:45:24 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:45:25.098574 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97342bde-ca56-4972-ac0b-885e413a5a74 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:25.100537 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-97342bde-ca56-4972-ac0b-885e413a5a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:25.165939 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97342bde-ca56-4972-ac0b-885e413a5a74 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:25.169842 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c11cbe-4e79-4066-8017-775618f1c4f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:25.170570 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c11cbe-4e79-4066-8017-775618f1c4f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:25.170925 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c11cbe-4e79-4066-8017-775618f1c4f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:25.171288 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c11cbe-4e79-4066-8017-775618f1c4f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:25.182767 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c11cbe-4e79-4066-8017-775618f1c4f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:25.184371 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c11cbe-4e79-4066-8017-775618f1c4f6 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:25.194623 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1393/2797] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:45:25 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:45:26.450728 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f22061-488f-461d-84f4-35e3c4aed02c None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:26.454006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-80f22061-488f-461d-84f4-35e3c4aed02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:26.534122 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f22061-488f-461d-84f4-35e3c4aed02c None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:26.538172 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04dd9a1-939a-415b-9730-361bad635788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:26.539470 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04dd9a1-939a-415b-9730-361bad635788 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:26.539777 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04dd9a1-939a-415b-9730-361bad635788 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:26.540049 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04dd9a1-939a-415b-9730-361bad635788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:26.541976 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e04dd9a1-939a-415b-9730-361bad635788 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:26.552688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04dd9a1-939a-415b-9730-361bad635788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:26.555138 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04dd9a1-939a-415b-9730-361bad635788 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:26.565696 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1405/2798] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:45:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:45:26.618789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c11cbe-4e79-4066-8017-775618f1c4f6 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:26.661612 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04dd9a1-939a-415b-9730-361bad635788 service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:26.664531 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e04dd9a1-939a-415b-9730-361bad635788 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:26.672031 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:26.676336 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04dd9a1-939a-415b-9730-361bad635788 service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:26.683317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58706ef2-ff42-446b-bec1-e6a07eb06999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:26.683963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58706ef2-ff42-446b-bec1-e6a07eb06999 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:26.684243 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58706ef2-ff42-446b-bec1-e6a07eb06999 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:26.684513 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58706ef2-ff42-446b-bec1-e6a07eb06999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:26.686379 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-58706ef2-ff42-446b-bec1-e6a07eb06999 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:26.700346 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c11cbe-4e79-4066-8017-775618f1c4f6 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:26.704907 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec302021-9f14-40e2-b76a-e13763c363b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:26.705858 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec302021-9f14-40e2-b76a-e13763c363b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:26.706067 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec302021-9f14-40e2-b76a-e13763c363b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:26.706344 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec302021-9f14-40e2-b76a-e13763c363b5 admin admin] PATH_INFO: `/v3/users/f41df8d3c64b45c695cb9420c9c22431` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:26.707675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec302021-9f14-40e2-b76a-e13763c363b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f41df8d3c64b45c695cb9420c9c22431)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:26.709537 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec302021-9f14-40e2-b76a-e13763c363b5 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:26.758464 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ec302021-9f14-40e2-b76a-e13763c363b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f41df8d3c64b45c695cb9420c9c22431'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:26.761761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58706ef2-ff42-446b-bec1-e6a07eb06999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:26.763317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58706ef2-ff42-446b-bec1-e6a07eb06999 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:26.766556 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ec302021-9f14-40e2-b76a-e13763c363b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f41df8d3c64b45c695cb9420c9c22431'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:26.767307 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ec302021-9f14-40e2-b76a-e13763c363b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f41df8d3c64b45c695cb9420c9c22431'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:26.768038 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ec302021-9f14-40e2-b76a-e13763c363b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f41df8d3c64b45c695cb9420c9c22431'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:26.776598 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1406/2799] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:45:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:45:26.784205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-ec302021-9f14-40e2-b76a-e13763c363b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f41df8d3c64b45c695cb9420c9c22431'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:26.796789 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1394/2800] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:45:26 2022] DELETE /identity/v3/users/f41df8d3c64b45c695cb9420c9c22431 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:45:26.801437 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd10037-58d0-4b67-9424-977bfb228eab None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:26.801886 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd10037-58d0-4b67-9424-977bfb228eab None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:26.802106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd10037-58d0-4b67-9424-977bfb228eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:26.819232 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-8bd10037-58d0-4b67-9424-977bfb228eab None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:45:26.821097 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-8bd10037-58d0-4b67-9424-977bfb228eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:26.837287 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec302021-9f14-40e2-b76a-e13763c363b5 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:26.838696 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-ec302021-9f14-40e2-b76a-e13763c363b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:26.857866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:26.886506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1407/2801] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:45:26.893259 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd10037-58d0-4b67-9424-977bfb228eab None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:26.895235 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-8bd10037-58d0-4b67-9424-977bfb228eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:26.914757 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec302021-9f14-40e2-b76a-e13763c363b5 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:26.920197 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e79f2d-21d9-4e66-a8ca-a9f5bd85b14d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afad68819504c2b864851c6989ae25a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:26.920849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e79f2d-21d9-4e66-a8ca-a9f5bd85b14d service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:26.921161 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e79f2d-21d9-4e66-a8ca-a9f5bd85b14d service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:26.921432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e79f2d-21d9-4e66-a8ca-a9f5bd85b14d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:26.923318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-60e79f2d-21d9-4e66-a8ca-a9f5bd85b14d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:26.935456 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e79f2d-21d9-4e66-a8ca-a9f5bd85b14d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:26.937033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e79f2d-21d9-4e66-a8ca-a9f5bd85b14d service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:26.950083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1395/2802] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:45:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:45:26.959582 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd10037-58d0-4b67-9424-977bfb228eab None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:26.964187 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39230360-832f-4eb9-be6f-35d867265c24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:26.965084 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39230360-832f-4eb9-be6f-35d867265c24 service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:26.965373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39230360-832f-4eb9-be6f-35d867265c24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:26.965651 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39230360-832f-4eb9-be6f-35d867265c24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:26.976536 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39230360-832f-4eb9-be6f-35d867265c24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:26.977980 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39230360-832f-4eb9-be6f-35d867265c24 service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:26.987726 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1408/2803] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:45:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:45:27.285344 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e79f2d-21d9-4e66-a8ca-a9f5bd85b14d service placement] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:27.287055 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-60e79f2d-21d9-4e66-a8ca-a9f5bd85b14d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:27.306753 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e79f2d-21d9-4e66-a8ca-a9f5bd85b14d service placement] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:27.313558 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e4ae3-f2b2-48ed-b007-37bbd493c994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:27.315083 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e4ae3-f2b2-48ed-b007-37bbd493c994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:27.315425 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e4ae3-f2b2-48ed-b007-37bbd493c994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:27.315773 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e4ae3-f2b2-48ed-b007-37bbd493c994 admin admin] PATH_INFO: `/v3/projects/147461e41352426493251ac0f6a65a1f` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:27.317243 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e4ae3-f2b2-48ed-b007-37bbd493c994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=147461e41352426493251ac0f6a65a1f)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:27.321302 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e4ae3-f2b2-48ed-b007-37bbd493c994 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:27.378330 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e22e4ae3-f2b2-48ed-b007-37bbd493c994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '147461e41352426493251ac0f6a65a1f', 'request_id': 'req-e22e4ae3-f2b2-48ed-b007-37bbd493c994'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:27.385193 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e22e4ae3-f2b2-48ed-b007-37bbd493c994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '147461e41352426493251ac0f6a65a1f', 'request_id': 'req-e22e4ae3-f2b2-48ed-b007-37bbd493c994'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:27.390672 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e22e4ae3-f2b2-48ed-b007-37bbd493c994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '147461e41352426493251ac0f6a65a1f', 'request_id': 'req-e22e4ae3-f2b2-48ed-b007-37bbd493c994'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:27.403547 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e22e4ae3-f2b2-48ed-b007-37bbd493c994 admin admin] The token cache is being invalidate because project 147461e41352426493251ac0f6a65a1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:45:27.403843 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e22e4ae3-f2b2-48ed-b007-37bbd493c994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:27.404450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-e22e4ae3-f2b2-48ed-b007-37bbd493c994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:27.405979 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1396/2804] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:45:27 2022] DELETE /identity/v3/projects/147461e41352426493251ac0f6a65a1f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:45:27.410027 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39230360-832f-4eb9-be6f-35d867265c24 service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:27.420478 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39230360-832f-4eb9-be6f-35d867265c24 service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:27.455285 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=100 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:27.483804 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c66f81-27d4-4322-a379-090d227b12db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:27.484752 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c66f81-27d4-4322-a379-090d227b12db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:27.485037 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c66f81-27d4-4322-a379-090d227b12db admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:27.485313 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c66f81-27d4-4322-a379-090d227b12db admin admin] PATH_INFO: `/v3/users/a5adba51e31c424ea8db196d59e389f2` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:27.486504 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c66f81-27d4-4322-a379-090d227b12db admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5adba51e31c424ea8db196d59e389f2)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:27.488538 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c66f81-27d4-4322-a379-090d227b12db admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:27.534740 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-84c66f81-27d4-4322-a379-090d227b12db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5adba51e31c424ea8db196d59e389f2'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:27.542041 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-84c66f81-27d4-4322-a379-090d227b12db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5adba51e31c424ea8db196d59e389f2'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:27.549821 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-84c66f81-27d4-4322-a379-090d227b12db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5adba51e31c424ea8db196d59e389f2'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:27.550469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-84c66f81-27d4-4322-a379-090d227b12db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5adba51e31c424ea8db196d59e389f2'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:27.551118 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-84c66f81-27d4-4322-a379-090d227b12db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5adba51e31c424ea8db196d59e389f2'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:27.568310 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1409/2805] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:45:27 2022] DELETE /identity/v3/users/a5adba51e31c424ea8db196d59e389f2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:45:27.949000 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e4ae3-f2b2-48ed-b007-37bbd493c994 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:27.950752 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-e22e4ae3-f2b2-48ed-b007-37bbd493c994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:27.967709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e4ae3-f2b2-48ed-b007-37bbd493c994 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:28.033283 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5941c-c391-454f-bcd2-e7235e772f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:28.034253 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5941c-c391-454f-bcd2-e7235e772f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:28.034557 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5941c-c391-454f-bcd2-e7235e772f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:28.034857 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5941c-c391-454f-bcd2-e7235e772f6d admin admin] PATH_INFO: `/v3/projects/54413de834e14bab983f236e6c3345d6` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:28.036221 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5941c-c391-454f-bcd2-e7235e772f6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=54413de834e14bab983f236e6c3345d6)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:28.038161 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5941c-c391-454f-bcd2-e7235e772f6d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:28.074580 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:28.102715 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-06a5941c-c391-454f-bcd2-e7235e772f6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54413de834e14bab983f236e6c3345d6', 'request_id': 'req-06a5941c-c391-454f-bcd2-e7235e772f6d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:28.109621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-06a5941c-c391-454f-bcd2-e7235e772f6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54413de834e14bab983f236e6c3345d6', 'request_id': 'req-06a5941c-c391-454f-bcd2-e7235e772f6d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:28.115432 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-06a5941c-c391-454f-bcd2-e7235e772f6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54413de834e14bab983f236e6c3345d6', 'request_id': 'req-06a5941c-c391-454f-bcd2-e7235e772f6d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:28.129447 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-06a5941c-c391-454f-bcd2-e7235e772f6d admin admin] The token cache is being invalidate because project 54413de834e14bab983f236e6c3345d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:45:28.129734 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-06a5941c-c391-454f-bcd2-e7235e772f6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:28.130344 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-06a5941c-c391-454f-bcd2-e7235e772f6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:28.131901 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1397/2806] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:45:27 2022] DELETE /identity/v3/projects/54413de834e14bab983f236e6c3345d6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:45:28.137549 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e219dfb-579d-47a1-a980-3ce733ff0eb0 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:28.137954 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e219dfb-579d-47a1-a980-3ce733ff0eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:28.138310 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e219dfb-579d-47a1-a980-3ce733ff0eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:28.157208 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-5e219dfb-579d-47a1-a980-3ce733ff0eb0 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:45:28.158445 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-5e219dfb-579d-47a1-a980-3ce733ff0eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:28.222734 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1410/2807] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:45:28.233204 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9736f6d-ce34-44a3-9729-64fd14877cfc None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:28.233496 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9736f6d-ce34-44a3-9729-64fd14877cfc None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:28.233765 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9736f6d-ce34-44a3-9729-64fd14877cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:28.252689 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-a9736f6d-ce34-44a3-9729-64fd14877cfc None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:45:28.253888 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a9736f6d-ce34-44a3-9729-64fd14877cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:28.261968 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1398/2808] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:45:28.266141 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e219dfb-579d-47a1-a980-3ce733ff0eb0 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:28.276476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e219dfb-579d-47a1-a980-3ce733ff0eb0 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:28.281239 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80331250-6bee-4d31-b418-0e8214dc90cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:28.281897 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80331250-6bee-4d31-b418-0e8214dc90cc admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:28.282182 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80331250-6bee-4d31-b418-0e8214dc90cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:28.282459 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80331250-6bee-4d31-b418-0e8214dc90cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:28.283222 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80331250-6bee-4d31-b418-0e8214dc90cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80178) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:45:28.283533 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80331250-6bee-4d31-b418-0e8214dc90cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:28.285116 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80331250-6bee-4d31-b418-0e8214dc90cc admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:28.297303 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1411/2809] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:45:28.301728 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9736f6d-ce34-44a3-9729-64fd14877cfc None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:28.311709 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9736f6d-ce34-44a3-9729-64fd14877cfc None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:28.316347 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d745cea-6fb5-45ff-8ee5-4dca422beff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:28.316985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d745cea-6fb5-45ff-8ee5-4dca422beff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:28.317264 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d745cea-6fb5-45ff-8ee5-4dca422beff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:28.317612 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d745cea-6fb5-45ff-8ee5-4dca422beff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:28.318228 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d745cea-6fb5-45ff-8ee5-4dca422beff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:28.320799 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d745cea-6fb5-45ff-8ee5-4dca422beff6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:28.348819 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1399/2810] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:45:28 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:45:28.352471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80331250-6bee-4d31-b418-0e8214dc90cc admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:28.362510 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80331250-6bee-4d31-b418-0e8214dc90cc admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:28.401423 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:28.421095 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e34d9-763f-4e7f-975c-793fbee1c336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:28.421786 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e34d9-763f-4e7f-975c-793fbee1c336 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:28.422067 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e34d9-763f-4e7f-975c-793fbee1c336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:28.422343 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e34d9-763f-4e7f-975c-793fbee1c336 admin admin] PATH_INFO: `/v3/users` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:28.423096 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e34d9-763f-4e7f-975c-793fbee1c336 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:28.425008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e34d9-763f-4e7f-975c-793fbee1c336 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:28.445168 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1412/2811] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:45:28 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:45:28.451924 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d745cea-6fb5-45ff-8ee5-4dca422beff6 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:28.464556 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d745cea-6fb5-45ff-8ee5-4dca422beff6 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:28.469385 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41473b31-c53f-476e-be14-a0ae74f505dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:28.470042 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41473b31-c53f-476e-be14-a0ae74f505dc admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:28.470324 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41473b31-c53f-476e-be14-a0ae74f505dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:28.470596 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41473b31-c53f-476e-be14-a0ae74f505dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:28.471354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41473b31-c53f-476e-be14-a0ae74f505dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:28.472996 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41473b31-c53f-476e-be14-a0ae74f505dc admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:28.484178 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1400/2812] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:45:28.488156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e34d9-763f-4e7f-975c-793fbee1c336 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:28.498720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e34d9-763f-4e7f-975c-793fbee1c336 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:28.503789 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31dff0-47d5-488c-a6a9-c26ee6b47bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:28.504463 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31dff0-47d5-488c-a6a9-c26ee6b47bae admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:28.504746 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31dff0-47d5-488c-a6a9-c26ee6b47bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:28.505021 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31dff0-47d5-488c-a6a9-c26ee6b47bae admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:28.505752 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a31dff0-47d5-488c-a6a9-c26ee6b47bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:28.507940 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a31dff0-47d5-488c-a6a9-c26ee6b47bae admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:28.518490 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1413/2813] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:45:28.523487 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41473b31-c53f-476e-be14-a0ae74f505dc admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:28.533713 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41473b31-c53f-476e-be14-a0ae74f505dc admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:28.538677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80752b7c-2c36-4d5f-8600-8a1c79273c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:28.539584 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80752b7c-2c36-4d5f-8600-8a1c79273c60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:28.539866 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80752b7c-2c36-4d5f-8600-8a1c79273c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:28.540139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80752b7c-2c36-4d5f-8600-8a1c79273c60 admin admin] PATH_INFO: `/v3/projects/058e02623f6d4f53b4dd6940e86117e8/users/c68b0efc058b4dfeb34d14b044602f5d/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:28.556450 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80752b7c-2c36-4d5f-8600-8a1c79273c60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=058e02623f6d4f53b4dd6940e86117e8, user_id=c68b0efc058b4dfeb34d14b044602f5d, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:28.559327 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80752b7c-2c36-4d5f-8600-8a1c79273c60 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:28.573760 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1401/2814] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:45:28 2022] PUT /identity/v3/projects/058e02623f6d4f53b4dd6940e86117e8/users/c68b0efc058b4dfeb34d14b044602f5d/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:45:28.579940 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7908bd-2d35-4750-98c0-a090c66ccc2a None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:28.580230 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7908bd-2d35-4750-98c0-a090c66ccc2a None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:28.580506 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7908bd-2d35-4750-98c0-a090c66ccc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:28.599979 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-4c7908bd-2d35-4750-98c0-a090c66ccc2a None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:45:28.601268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4c7908bd-2d35-4750-98c0-a090c66ccc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:28.672132 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1414/2815] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:45:28.684698 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80752b7c-2c36-4d5f-8600-8a1c79273c60 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:28.686477 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-80752b7c-2c36-4d5f-8600-8a1c79273c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:28.745653 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80752b7c-2c36-4d5f-8600-8a1c79273c60 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:28.749338 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600df70-65c7-40de-90ce-cd646407256e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:28.749990 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600df70-65c7-40de-90ce-cd646407256e service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:28.750268 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600df70-65c7-40de-90ce-cd646407256e service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:28.750538 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600df70-65c7-40de-90ce-cd646407256e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:28.761267 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600df70-65c7-40de-90ce-cd646407256e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:28.762903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600df70-65c7-40de-90ce-cd646407256e service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:28.775022 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1402/2816] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:45:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:45:29.053629 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7908bd-2d35-4750-98c0-a090c66ccc2a None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:29.055430 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4c7908bd-2d35-4750-98c0-a090c66ccc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:29.152602 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7908bd-2d35-4750-98c0-a090c66ccc2a None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:29.159073 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac740f12-48c3-4672-9ca6-e72c56dab3fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:29.159834 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac740f12-48c3-4672-9ca6-e72c56dab3fb service nova] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:29.160121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac740f12-48c3-4672-9ca6-e72c56dab3fb service nova] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:29.160399 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac740f12-48c3-4672-9ca6-e72c56dab3fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:29.162297 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-ac740f12-48c3-4672-9ca6-e72c56dab3fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:29.175765 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac740f12-48c3-4672-9ca6-e72c56dab3fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:29.177356 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac740f12-48c3-4672-9ca6-e72c56dab3fb service nova] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:29.189086 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1415/2817] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:45:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:45:29.954670 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3caeca5-3b32-415a-ae2a-da7294ccdee2 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:29.955224 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3caeca5-3b32-415a-ae2a-da7294ccdee2 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:29.955342 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3caeca5-3b32-415a-ae2a-da7294ccdee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:29.983795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-d3caeca5-3b32-415a-ae2a-da7294ccdee2 None None] MFA Rules not processed for user `c68b0efc058b4dfeb34d14b044602f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:45:29.985027 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-d3caeca5-3b32-415a-ae2a-da7294ccdee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:30.033275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:30.056377 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1403/2818] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:45:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:45:30.061738 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac740f12-48c3-4672-9ca6-e72c56dab3fb service nova] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:30.073163 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac740f12-48c3-4672-9ca6-e72c56dab3fb service nova] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:30.077792 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb5d60-a905-4c8b-9499-520a29e3b9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:30.078450 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb5d60-a905-4c8b-9499-520a29e3b9e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:30.078781 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb5d60-a905-4c8b-9499-520a29e3b9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:30.079083 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb5d60-a905-4c8b-9499-520a29e3b9e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:30.079791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb5d60-a905-4c8b-9499-520a29e3b9e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:30.081680 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb5d60-a905-4c8b-9499-520a29e3b9e5 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:30.113774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1416/2819] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:45:30 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:45:30.118231 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3caeca5-3b32-415a-ae2a-da7294ccdee2 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:30.129154 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3caeca5-3b32-415a-ae2a-da7294ccdee2 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:30.187972 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97974fe-7f89-470d-aa28-a74d5bc63629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:30.188655 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97974fe-7f89-470d-aa28-a74d5bc63629 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:30.188939 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97974fe-7f89-470d-aa28-a74d5bc63629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:30.189211 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97974fe-7f89-470d-aa28-a74d5bc63629 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:30.189938 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97974fe-7f89-470d-aa28-a74d5bc63629 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:30.191895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97974fe-7f89-470d-aa28-a74d5bc63629 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:30.213251 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1404/2820] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:45:30 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 20:45:30.216352 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb5d60-a905-4c8b-9499-520a29e3b9e5 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:30.227130 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb5d60-a905-4c8b-9499-520a29e3b9e5 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:30.231724 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c5db6-7442-4134-b289-5dc560131cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:30.232386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c5db6-7442-4134-b289-5dc560131cca admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:30.232664 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c5db6-7442-4134-b289-5dc560131cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:30.232937 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c5db6-7442-4134-b289-5dc560131cca admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:30.233663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c5db6-7442-4134-b289-5dc560131cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:30.235280 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c5db6-7442-4134-b289-5dc560131cca admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:30.246602 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1417/2821] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:45:30.251007 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97974fe-7f89-470d-aa28-a74d5bc63629 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:30.261958 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97974fe-7f89-470d-aa28-a74d5bc63629 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:30.267341 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4bbbf-a0da-4efa-ae53-29744f40f69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:30.268256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4bbbf-a0da-4efa-ae53-29744f40f69c admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:30.268534 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4bbbf-a0da-4efa-ae53-29744f40f69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:30.268812 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4bbbf-a0da-4efa-ae53-29744f40f69c admin admin] PATH_INFO: `/v3/projects/389e4e5addb44c588ec3fc42ed9ccd11/users/59b198e15f734f06b6330ec1547e7d10/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:30.285916 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4bbbf-a0da-4efa-ae53-29744f40f69c admin admin] RBAC: Authorizing `identity:create_grant(project_id=389e4e5addb44c588ec3fc42ed9ccd11, user_id=59b198e15f734f06b6330ec1547e7d10, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:30.288673 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4bbbf-a0da-4efa-ae53-29744f40f69c admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:30.306183 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1405/2822] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:45:30 2022] PUT /identity/v3/projects/389e4e5addb44c588ec3fc42ed9ccd11/users/59b198e15f734f06b6330ec1547e7d10/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:45:30.307409 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c5db6-7442-4134-b289-5dc560131cca admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:30.317700 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c5db6-7442-4134-b289-5dc560131cca admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:30.326762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:30.381566 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb22039-aac6-44a6-90f8-54c2dfec4b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:30.383576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb22039-aac6-44a6-90f8-54c2dfec4b53 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:30.383864 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb22039-aac6-44a6-90f8-54c2dfec4b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:30.384140 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb22039-aac6-44a6-90f8-54c2dfec4b53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:30.384948 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb22039-aac6-44a6-90f8-54c2dfec4b53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:30.387237 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb22039-aac6-44a6-90f8-54c2dfec4b53 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:30.402156 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1418/2823] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:45:30.404750 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4bbbf-a0da-4efa-ae53-29744f40f69c admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:30.417379 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4bbbf-a0da-4efa-ae53-29744f40f69c admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:30.422675 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38ab47-99c1-4c48-b24a-8c469a923ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:30.423626 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38ab47-99c1-4c48-b24a-8c469a923ef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:30.423904 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38ab47-99c1-4c48-b24a-8c469a923ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:30.424217 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38ab47-99c1-4c48-b24a-8c469a923ef4 admin admin] PATH_INFO: `/v3/projects/389e4e5addb44c588ec3fc42ed9ccd11/users/59b198e15f734f06b6330ec1547e7d10/roles/5ca112885467422684e134cb0f94865a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:30.426293 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38ab47-99c1-4c48-b24a-8c469a923ef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=389e4e5addb44c588ec3fc42ed9ccd11, user_id=59b198e15f734f06b6330ec1547e7d10, role_id=5ca112885467422684e134cb0f94865a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:30.429651 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38ab47-99c1-4c48-b24a-8c469a923ef4 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:30.440852 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1406/2824] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:45:30 2022] PUT /identity/v3/projects/389e4e5addb44c588ec3fc42ed9ccd11/users/59b198e15f734f06b6330ec1547e7d10/roles/5ca112885467422684e134cb0f94865a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:45:31.356039 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0fc8b-66c8-4cfa-ae3e-df116bf84507 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:31.356566 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0fc8b-66c8-4cfa-ae3e-df116bf84507 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:31.356787 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0fc8b-66c8-4cfa-ae3e-df116bf84507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:31.384906 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-9dd0fc8b-66c8-4cfa-ae3e-df116bf84507 None None] MFA Rules not processed for user `59b198e15f734f06b6330ec1547e7d10`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:45:31.386137 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9dd0fc8b-66c8-4cfa-ae3e-df116bf84507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:31.469734 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1419/2825] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:45:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:45:31.481882 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38ab47-99c1-4c48-b24a-8c469a923ef4 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:31.483934 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7e38ab47-99c1-4c48-b24a-8c469a923ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:31.560867 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38ab47-99c1-4c48-b24a-8c469a923ef4 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:31.564567 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34281c0-3db9-4ccb-988b-81e39083cc50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:31.565227 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34281c0-3db9-4ccb-988b-81e39083cc50 service cinder] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:31.565531 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34281c0-3db9-4ccb-988b-81e39083cc50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:31.565783 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34281c0-3db9-4ccb-988b-81e39083cc50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:31.623251 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:31.628864 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34281c0-3db9-4ccb-988b-81e39083cc50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:31.630761 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34281c0-3db9-4ccb-988b-81e39083cc50 service cinder] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:31.641413 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1407/2826] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:45:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:45:32.997151 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0fc8b-66c8-4cfa-ae3e-df116bf84507 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:33.000178 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9dd0fc8b-66c8-4cfa-ae3e-df116bf84507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:33.014225 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0fc8b-66c8-4cfa-ae3e-df116bf84507 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:33.017805 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90213ef-3683-42b9-a938-575e189c69c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:33.018474 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90213ef-3683-42b9-a938-575e189c69c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:33.018761 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90213ef-3683-42b9-a938-575e189c69c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:33.019072 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90213ef-3683-42b9-a938-575e189c69c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:33.029227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90213ef-3683-42b9-a938-575e189c69c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:33.030640 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90213ef-3683-42b9-a938-575e189c69c0 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:33.040292 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1420/2827] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:45:32 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:45:41.636987 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:43.035953 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=64 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:46.151102 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34281c0-3db9-4ccb-988b-81e39083cc50 service cinder] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:46.214680 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34281c0-3db9-4ccb-988b-81e39083cc50 service cinder] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:46.220239 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae52652-64c9-4288-95b4-e3178452f0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:46.221166 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae52652-64c9-4288-95b4-e3178452f0f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:46.221455 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae52652-64c9-4288-95b4-e3178452f0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:46.221731 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae52652-64c9-4288-95b4-e3178452f0f2 admin admin] PATH_INFO: `/v3/users/c68b0efc058b4dfeb34d14b044602f5d` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:46.222992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae52652-64c9-4288-95b4-e3178452f0f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c68b0efc058b4dfeb34d14b044602f5d)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:46.224939 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae52652-64c9-4288-95b4-e3178452f0f2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:46.277957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4ae52652-64c9-4288-95b4-e3178452f0f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c68b0efc058b4dfeb34d14b044602f5d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:46.286317 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4ae52652-64c9-4288-95b4-e3178452f0f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c68b0efc058b4dfeb34d14b044602f5d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:46.287030 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4ae52652-64c9-4288-95b4-e3178452f0f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c68b0efc058b4dfeb34d14b044602f5d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:46.287729 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4ae52652-64c9-4288-95b4-e3178452f0f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c68b0efc058b4dfeb34d14b044602f5d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:46.301861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-4ae52652-64c9-4288-95b4-e3178452f0f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c68b0efc058b4dfeb34d14b044602f5d'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:46.315684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1408/2828] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:45:46 2022] DELETE /identity/v3/users/c68b0efc058b4dfeb34d14b044602f5d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 20:45:46.512084 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90213ef-3683-42b9-a938-575e189c69c0 service cinder] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:46.525274 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90213ef-3683-42b9-a938-575e189c69c0 service cinder] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:46.586209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7fe9c-47f6-4dd7-9988-0a69d60ba6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:46.587061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7fe9c-47f6-4dd7-9988-0a69d60ba6ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:46.587345 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7fe9c-47f6-4dd7-9988-0a69d60ba6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:46.587645 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7fe9c-47f6-4dd7-9988-0a69d60ba6ff admin admin] PATH_INFO: `/v3/projects/058e02623f6d4f53b4dd6940e86117e8` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:46.588806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7fe9c-47f6-4dd7-9988-0a69d60ba6ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=058e02623f6d4f53b4dd6940e86117e8)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:46.590746 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7fe9c-47f6-4dd7-9988-0a69d60ba6ff admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:46.644930 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-15d7fe9c-47f6-4dd7-9988-0a69d60ba6ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '058e02623f6d4f53b4dd6940e86117e8', 'request_id': 'req-15d7fe9c-47f6-4dd7-9988-0a69d60ba6ff'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:46.652920 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-15d7fe9c-47f6-4dd7-9988-0a69d60ba6ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '058e02623f6d4f53b4dd6940e86117e8', 'request_id': 'req-15d7fe9c-47f6-4dd7-9988-0a69d60ba6ff'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:46.659471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-15d7fe9c-47f6-4dd7-9988-0a69d60ba6ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '058e02623f6d4f53b4dd6940e86117e8', 'request_id': 'req-15d7fe9c-47f6-4dd7-9988-0a69d60ba6ff'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:46.671206 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-15d7fe9c-47f6-4dd7-9988-0a69d60ba6ff admin admin] The token cache is being invalidate because project 058e02623f6d4f53b4dd6940e86117e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:45:46.671499 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-15d7fe9c-47f6-4dd7-9988-0a69d60ba6ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:46.672163 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-15d7fe9c-47f6-4dd7-9988-0a69d60ba6ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:46.673762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1421/2829] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:45:46 2022] DELETE /identity/v3/projects/058e02623f6d4f53b4dd6940e86117e8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:45:46.677941 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae52652-64c9-4288-95b4-e3178452f0f2 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:46.679595 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4ae52652-64c9-4288-95b4-e3178452f0f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:46.694527 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae52652-64c9-4288-95b4-e3178452f0f2 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:46.752029 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2f1d2-6d3b-4575-8fa4-9650b9a5f983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:46.752910 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2f1d2-6d3b-4575-8fa4-9650b9a5f983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:46.753196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2f1d2-6d3b-4575-8fa4-9650b9a5f983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:46.753473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2f1d2-6d3b-4575-8fa4-9650b9a5f983 admin admin] PATH_INFO: `/v3/users/59b198e15f734f06b6330ec1547e7d10` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:46.754726 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2f1d2-6d3b-4575-8fa4-9650b9a5f983 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59b198e15f734f06b6330ec1547e7d10)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:46.756796 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2f1d2-6d3b-4575-8fa4-9650b9a5f983 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:46.778960 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:46.807139 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7be2f1d2-6d3b-4575-8fa4-9650b9a5f983 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59b198e15f734f06b6330ec1547e7d10'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:46.814928 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7be2f1d2-6d3b-4575-8fa4-9650b9a5f983 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59b198e15f734f06b6330ec1547e7d10'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:46.815673 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7be2f1d2-6d3b-4575-8fa4-9650b9a5f983 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59b198e15f734f06b6330ec1547e7d10'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:46.816219 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7be2f1d2-6d3b-4575-8fa4-9650b9a5f983 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59b198e15f734f06b6330ec1547e7d10'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:46.828567 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7be2f1d2-6d3b-4575-8fa4-9650b9a5f983 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59b198e15f734f06b6330ec1547e7d10'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:46.841787 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1409/2830] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:45:46 2022] DELETE /identity/v3/users/59b198e15f734f06b6330ec1547e7d10 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:45:47.032841 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7fe9c-47f6-4dd7-9988-0a69d60ba6ff admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:47.034314 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-15d7fe9c-47f6-4dd7-9988-0a69d60ba6ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:47.047394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7fe9c-47f6-4dd7-9988-0a69d60ba6ff admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:47.113316 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8cb42-1b81-42b5-a628-4296b436cb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:47.114407 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8cb42-1b81-42b5-a628-4296b436cb1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:47.114851 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8cb42-1b81-42b5-a628-4296b436cb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:47.115250 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8cb42-1b81-42b5-a628-4296b436cb1a admin admin] PATH_INFO: `/v3/projects/389e4e5addb44c588ec3fc42ed9ccd11` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:47.116688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8cb42-1b81-42b5-a628-4296b436cb1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=389e4e5addb44c588ec3fc42ed9ccd11)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:47.120765 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8cb42-1b81-42b5-a628-4296b436cb1a admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:47.164505 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:47.178052 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8be8cb42-1b81-42b5-a628-4296b436cb1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '389e4e5addb44c588ec3fc42ed9ccd11', 'request_id': 'req-8be8cb42-1b81-42b5-a628-4296b436cb1a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:47.186097 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8be8cb42-1b81-42b5-a628-4296b436cb1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '389e4e5addb44c588ec3fc42ed9ccd11', 'request_id': 'req-8be8cb42-1b81-42b5-a628-4296b436cb1a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:47.192448 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8be8cb42-1b81-42b5-a628-4296b436cb1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '389e4e5addb44c588ec3fc42ed9ccd11', 'request_id': 'req-8be8cb42-1b81-42b5-a628-4296b436cb1a'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:47.204363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8be8cb42-1b81-42b5-a628-4296b436cb1a admin admin] The token cache is being invalidate because project 389e4e5addb44c588ec3fc42ed9ccd11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:45:47.204933 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8be8cb42-1b81-42b5-a628-4296b436cb1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:47.205317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-8be8cb42-1b81-42b5-a628-4296b436cb1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:47.206959 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1422/2831] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:45:47 2022] DELETE /identity/v3/projects/389e4e5addb44c588ec3fc42ed9ccd11 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:45:47.212018 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1006d-e30d-47fc-95cd-220b9579f0d1 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:47.212519 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1006d-e30d-47fc-95cd-220b9579f0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:47.212621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1006d-e30d-47fc-95cd-220b9579f0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:47.234575 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-8bb1006d-e30d-47fc-95cd-220b9579f0d1 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:45:47.235839 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-8bb1006d-e30d-47fc-95cd-220b9579f0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:47.297210 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1410/2832] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:45:47.306674 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce54038-bbdd-416a-ae27-1a1936692b4a None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:47.307061 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce54038-bbdd-416a-ae27-1a1936692b4a None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:47.307362 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce54038-bbdd-416a-ae27-1a1936692b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:47.328530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-2ce54038-bbdd-416a-ae27-1a1936692b4a None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:45:47.329790 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-2ce54038-bbdd-416a-ae27-1a1936692b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:47.338965 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1423/2833] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:45:47.343647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1006d-e30d-47fc-95cd-220b9579f0d1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:47.353629 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1006d-e30d-47fc-95cd-220b9579f0d1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:47.358427 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b477a06-bd9e-4165-a32d-c5932cd622d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:47.359122 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b477a06-bd9e-4165-a32d-c5932cd622d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:47.359410 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b477a06-bd9e-4165-a32d-c5932cd622d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:47.359688 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b477a06-bd9e-4165-a32d-c5932cd622d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:47.360448 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b477a06-bd9e-4165-a32d-c5932cd622d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:45:47.360755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b477a06-bd9e-4165-a32d-c5932cd622d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:47.362348 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b477a06-bd9e-4165-a32d-c5932cd622d7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:47.374621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1411/2834] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:45:47.379411 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce54038-bbdd-416a-ae27-1a1936692b4a None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:47.391246 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce54038-bbdd-416a-ae27-1a1936692b4a None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:47.396903 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fdc6e-98c2-4e52-a7c3-5e007f33170f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:47.397561 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fdc6e-98c2-4e52-a7c3-5e007f33170f admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:47.397908 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fdc6e-98c2-4e52-a7c3-5e007f33170f admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:47.398272 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fdc6e-98c2-4e52-a7c3-5e007f33170f admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:47.399134 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9fdc6e-98c2-4e52-a7c3-5e007f33170f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:47.401269 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9fdc6e-98c2-4e52-a7c3-5e007f33170f admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:47.426428 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1424/2835] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:45:47 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:45:47.432203 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b477a06-bd9e-4165-a32d-c5932cd622d7 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:47.443958 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b477a06-bd9e-4165-a32d-c5932cd622d7 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:47.507623 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e26ee-b8cc-4088-ad7a-4d7acae33d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:47.508797 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e26ee-b8cc-4088-ad7a-4d7acae33d4d admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:47.509080 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e26ee-b8cc-4088-ad7a-4d7acae33d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:47.509354 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e26ee-b8cc-4088-ad7a-4d7acae33d4d admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:47.510035 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e26ee-b8cc-4088-ad7a-4d7acae33d4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:47.511995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e26ee-b8cc-4088-ad7a-4d7acae33d4d admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:47.532562 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1412/2836] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:45:47 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:45:47.536895 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fdc6e-98c2-4e52-a7c3-5e007f33170f admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:47.547146 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fdc6e-98c2-4e52-a7c3-5e007f33170f admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:47.551747 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf3178-4b94-4735-bd7d-07b5de66edf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:47.552404 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf3178-4b94-4735-bd7d-07b5de66edf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:47.552688 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf3178-4b94-4735-bd7d-07b5de66edf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:47.552963 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf3178-4b94-4735-bd7d-07b5de66edf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:47.553694 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf3178-4b94-4735-bd7d-07b5de66edf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:47.555817 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf3178-4b94-4735-bd7d-07b5de66edf9 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:47.566101 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1425/2837] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:45:47.570729 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e26ee-b8cc-4088-ad7a-4d7acae33d4d admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:47.581063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e26ee-b8cc-4088-ad7a-4d7acae33d4d admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:47.585881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ccf10-00c6-43fd-981a-e9adf599ac22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:47.586550 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ccf10-00c6-43fd-981a-e9adf599ac22 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:47.586835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ccf10-00c6-43fd-981a-e9adf599ac22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:47.587142 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ccf10-00c6-43fd-981a-e9adf599ac22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:47.587873 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ccf10-00c6-43fd-981a-e9adf599ac22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:47.589473 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ccf10-00c6-43fd-981a-e9adf599ac22 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:47.600341 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1413/2838] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:45:47.605245 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf3178-4b94-4735-bd7d-07b5de66edf9 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:47.615460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf3178-4b94-4735-bd7d-07b5de66edf9 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:47.621647 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f83c70-133c-4af0-b44b-a50db2e14d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:47.622682 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f83c70-133c-4af0-b44b-a50db2e14d05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:47.623060 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f83c70-133c-4af0-b44b-a50db2e14d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:47.623387 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f83c70-133c-4af0-b44b-a50db2e14d05 admin admin] PATH_INFO: `/v3/projects/052bd3986862406297a6e87da4917a78/users/02b316e2163641f9b6b9c9b3a802a9f5/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:47.641547 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f83c70-133c-4af0-b44b-a50db2e14d05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=052bd3986862406297a6e87da4917a78, user_id=02b316e2163641f9b6b9c9b3a802a9f5, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:47.644954 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f83c70-133c-4af0-b44b-a50db2e14d05 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:47.659241 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1426/2839] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:45:47 2022] PUT /identity/v3/projects/052bd3986862406297a6e87da4917a78/users/02b316e2163641f9b6b9c9b3a802a9f5/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:45:47.670196 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ff305-a6ba-4ab0-b0c6-f7c6cf0aaa06 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:47.670488 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ff305-a6ba-4ab0-b0c6-f7c6cf0aaa06 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:47.670760 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ff305-a6ba-4ab0-b0c6-f7c6cf0aaa06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:47.677717 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:47.701399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-b42ff305-a6ba-4ab0-b0c6-f7c6cf0aaa06 None None] MFA Rules not processed for user `02b316e2163641f9b6b9c9b3a802a9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:45:47.703444 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-b42ff305-a6ba-4ab0-b0c6-f7c6cf0aaa06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:47.770394 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1414/2840] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:45:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:45:47.782645 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f83c70-133c-4af0-b44b-a50db2e14d05 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:47.784443 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-00f83c70-133c-4af0-b44b-a50db2e14d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:47.843080 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f83c70-133c-4af0-b44b-a50db2e14d05 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:47.846971 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97096a4f-044a-41cf-bf46-79d7b7fd612a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:47.847661 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97096a4f-044a-41cf-bf46-79d7b7fd612a service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:47.847949 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97096a4f-044a-41cf-bf46-79d7b7fd612a service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:47.848229 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97096a4f-044a-41cf-bf46-79d7b7fd612a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:47.857459 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97096a4f-044a-41cf-bf46-79d7b7fd612a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:47.858890 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97096a4f-044a-41cf-bf46-79d7b7fd612a service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:47.868677 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1427/2841] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:45:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:45:47.888539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ff305-a6ba-4ab0-b0c6-f7c6cf0aaa06 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:47.959193 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ff305-a6ba-4ab0-b0c6-f7c6cf0aaa06 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:47.964513 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a06ca9-d062-44ab-bb0d-12cb5dbed658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:47.965850 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a06ca9-d062-44ab-bb0d-12cb5dbed658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:47.965850 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a06ca9-d062-44ab-bb0d-12cb5dbed658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:47.966091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a06ca9-d062-44ab-bb0d-12cb5dbed658 admin admin] PATH_INFO: `/v3/users/02b316e2163641f9b6b9c9b3a802a9f5` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:47.967401 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a06ca9-d062-44ab-bb0d-12cb5dbed658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02b316e2163641f9b6b9c9b3a802a9f5)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:47.969341 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a06ca9-d062-44ab-bb0d-12cb5dbed658 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:48.019812 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-85a06ca9-d062-44ab-bb0d-12cb5dbed658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02b316e2163641f9b6b9c9b3a802a9f5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:48.027793 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-85a06ca9-d062-44ab-bb0d-12cb5dbed658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02b316e2163641f9b6b9c9b3a802a9f5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:48.028590 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-85a06ca9-d062-44ab-bb0d-12cb5dbed658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02b316e2163641f9b6b9c9b3a802a9f5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:48.029444 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-85a06ca9-d062-44ab-bb0d-12cb5dbed658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02b316e2163641f9b6b9c9b3a802a9f5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:48.044109 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-85a06ca9-d062-44ab-bb0d-12cb5dbed658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02b316e2163641f9b6b9c9b3a802a9f5'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:48.056376 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1415/2842] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:45:47 2022] DELETE /identity/v3/users/02b316e2163641f9b6b9c9b3a802a9f5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:45:48.060933 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335e7c3-6519-49f9-a7b0-6b272ec895c8 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:48.061220 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335e7c3-6519-49f9-a7b0-6b272ec895c8 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:48.061500 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335e7c3-6519-49f9-a7b0-6b272ec895c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:48.079146 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-9335e7c3-6519-49f9-a7b0-6b272ec895c8 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:45:48.080363 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9335e7c3-6519-49f9-a7b0-6b272ec895c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:48.132929 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:48.144194 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1428/2843] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:45:48.154414 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a06ca9-d062-44ab-bb0d-12cb5dbed658 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:48.156066 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-85a06ca9-d062-44ab-bb0d-12cb5dbed658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:48.171363 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:48.216854 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a06ca9-d062-44ab-bb0d-12cb5dbed658 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:48.220681 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5303b37-ea9e-4cd1-8e30-0559374f64fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:48.221368 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5303b37-ea9e-4cd1-8e30-0559374f64fa service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:48.221648 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5303b37-ea9e-4cd1-8e30-0559374f64fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:48.221920 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5303b37-ea9e-4cd1-8e30-0559374f64fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:48.232256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5303b37-ea9e-4cd1-8e30-0559374f64fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:48.233825 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5303b37-ea9e-4cd1-8e30-0559374f64fa service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:48.246536 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1416/2844] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:45:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:45:48.528729 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335e7c3-6519-49f9-a7b0-6b272ec895c8 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:48.530369 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9335e7c3-6519-49f9-a7b0-6b272ec895c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:48.544175 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335e7c3-6519-49f9-a7b0-6b272ec895c8 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:48.549161 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e63a96-9dab-4613-b291-be90ba84ca30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:48.551109 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e63a96-9dab-4613-b291-be90ba84ca30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:48.551907 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e63a96-9dab-4613-b291-be90ba84ca30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:48.552353 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e63a96-9dab-4613-b291-be90ba84ca30 admin admin] PATH_INFO: `/v3/projects/052bd3986862406297a6e87da4917a78` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:48.553906 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e63a96-9dab-4613-b291-be90ba84ca30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=052bd3986862406297a6e87da4917a78)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:48.556293 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e63a96-9dab-4613-b291-be90ba84ca30 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:48.608340 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-16e63a96-9dab-4613-b291-be90ba84ca30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '052bd3986862406297a6e87da4917a78', 'request_id': 'req-16e63a96-9dab-4613-b291-be90ba84ca30'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:48.616471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-16e63a96-9dab-4613-b291-be90ba84ca30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '052bd3986862406297a6e87da4917a78', 'request_id': 'req-16e63a96-9dab-4613-b291-be90ba84ca30'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:48.624855 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-16e63a96-9dab-4613-b291-be90ba84ca30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '052bd3986862406297a6e87da4917a78', 'request_id': 'req-16e63a96-9dab-4613-b291-be90ba84ca30'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:48.640245 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-16e63a96-9dab-4613-b291-be90ba84ca30 admin admin] The token cache is being invalidate because project 052bd3986862406297a6e87da4917a78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:45:48.640544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-16e63a96-9dab-4613-b291-be90ba84ca30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:48.641268 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-16e63a96-9dab-4613-b291-be90ba84ca30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:45:48.643070 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1429/2845] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:45:48 2022] DELETE /identity/v3/projects/052bd3986862406297a6e87da4917a78 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:45:48.648838 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09662da1-04bf-4a05-bc1b-3b9a309154d1 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:48.649136 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09662da1-04bf-4a05-bc1b-3b9a309154d1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:48.649417 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09662da1-04bf-4a05-bc1b-3b9a309154d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:48.667743 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-09662da1-04bf-4a05-bc1b-3b9a309154d1 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:45:48.668964 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-09662da1-04bf-4a05-bc1b-3b9a309154d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:48.731222 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1417/2846] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:45:48.739853 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be0aa7-8114-4f2e-88e7-74ffd8c6e5a6 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:48.740163 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be0aa7-8114-4f2e-88e7-74ffd8c6e5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:48.740438 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be0aa7-8114-4f2e-88e7-74ffd8c6e5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:48.760649 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-90be0aa7-8114-4f2e-88e7-74ffd8c6e5a6 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:45:48.761939 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-90be0aa7-8114-4f2e-88e7-74ffd8c6e5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:48.772300 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1430/2847] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:45:48.777141 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09662da1-04bf-4a05-bc1b-3b9a309154d1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:48.786906 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09662da1-04bf-4a05-bc1b-3b9a309154d1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:48.791600 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf85338-bdc6-4b73-a058-b64b5a2c1f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:48.792250 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf85338-bdc6-4b73-a058-b64b5a2c1f28 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:48.792535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf85338-bdc6-4b73-a058-b64b5a2c1f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:48.792810 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf85338-bdc6-4b73-a058-b64b5a2c1f28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:48.793532 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf85338-bdc6-4b73-a058-b64b5a2c1f28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:45:48.793835 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf85338-bdc6-4b73-a058-b64b5a2c1f28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:48.795494 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf85338-bdc6-4b73-a058-b64b5a2c1f28 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:48.808023 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1418/2848] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:45:48.812751 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be0aa7-8114-4f2e-88e7-74ffd8c6e5a6 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:48.823633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be0aa7-8114-4f2e-88e7-74ffd8c6e5a6 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:48.828558 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04314d74-ec22-4925-93dc-a467f092b070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:48.829251 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04314d74-ec22-4925-93dc-a467f092b070 admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:48.829536 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04314d74-ec22-4925-93dc-a467f092b070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:48.829813 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04314d74-ec22-4925-93dc-a467f092b070 admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:48.830547 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04314d74-ec22-4925-93dc-a467f092b070 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:48.832544 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04314d74-ec22-4925-93dc-a467f092b070 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:48.861770 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1431/2849] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:45:48 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:45:48.872955 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf85338-bdc6-4b73-a058-b64b5a2c1f28 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:48.884883 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf85338-bdc6-4b73-a058-b64b5a2c1f28 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:48.950189 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df5f2c-7669-452f-b396-84189e4c81ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:48.951134 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df5f2c-7669-452f-b396-84189e4c81ae admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:48.951572 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df5f2c-7669-452f-b396-84189e4c81ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:48.951979 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df5f2c-7669-452f-b396-84189e4c81ae admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:48.953080 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df5f2c-7669-452f-b396-84189e4c81ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:48.956253 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df5f2c-7669-452f-b396-84189e4c81ae admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:48.980507 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1419/2850] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:45:48 2022] POST /identity/v3/users => generated 285 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:45:48.984607 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04314d74-ec22-4925-93dc-a467f092b070 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:48.995373 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04314d74-ec22-4925-93dc-a467f092b070 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:49.000209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a877b-0357-4e5a-b38d-27d5b3f481d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:49.000863 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a877b-0357-4e5a-b38d-27d5b3f481d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:49.001147 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a877b-0357-4e5a-b38d-27d5b3f481d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:49.001428 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a877b-0357-4e5a-b38d-27d5b3f481d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:49.002162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a877b-0357-4e5a-b38d-27d5b3f481d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:49.003806 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a877b-0357-4e5a-b38d-27d5b3f481d1 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:49.014860 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1432/2851] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:45:49.018905 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df5f2c-7669-452f-b396-84189e4c81ae admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:49.029190 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df5f2c-7669-452f-b396-84189e4c81ae admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:49.033811 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298ad39-f791-407a-9528-2ba41912abe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:49.034465 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298ad39-f791-407a-9528-2ba41912abe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:49.034751 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298ad39-f791-407a-9528-2ba41912abe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:49.035063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298ad39-f791-407a-9528-2ba41912abe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:49.035796 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298ad39-f791-407a-9528-2ba41912abe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:49.037382 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298ad39-f791-407a-9528-2ba41912abe6 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:49.047399 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1420/2852] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:45:49.052126 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a877b-0357-4e5a-b38d-27d5b3f481d1 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:49.065658 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a877b-0357-4e5a-b38d-27d5b3f481d1 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:49.071023 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466ff44-2be0-48c5-8ae1-716474457f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:49.071909 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466ff44-2be0-48c5-8ae1-716474457f16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:49.072191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466ff44-2be0-48c5-8ae1-716474457f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:49.072465 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466ff44-2be0-48c5-8ae1-716474457f16 admin admin] PATH_INFO: `/v3/projects/296bf24530494ecfa32be16d52e6b1b8/users/b2a3c0c826294d088634f62dcc00c9bb/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:49.087134 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466ff44-2be0-48c5-8ae1-716474457f16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=296bf24530494ecfa32be16d52e6b1b8, user_id=b2a3c0c826294d088634f62dcc00c9bb, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:49.089778 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466ff44-2be0-48c5-8ae1-716474457f16 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:49.105796 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1433/2853] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:45:49 2022] PUT /identity/v3/projects/296bf24530494ecfa32be16d52e6b1b8/users/b2a3c0c826294d088634f62dcc00c9bb/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:45:49.116070 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d0699-8930-426a-a6d6-593b8dfa0cc5 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:49.116361 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d0699-8930-426a-a6d6-593b8dfa0cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:49.116684 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d0699-8930-426a-a6d6-593b8dfa0cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:49.149180 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-851d0699-8930-426a-a6d6-593b8dfa0cc5 None None] MFA Rules not processed for user `b2a3c0c826294d088634f62dcc00c9bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:45:49.150521 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-851d0699-8930-426a-a6d6-593b8dfa0cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:49.161381 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:49.217159 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1421/2854] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:45:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:45:49.230869 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466ff44-2be0-48c5-8ae1-716474457f16 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:45:49.232401 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-e466ff44-2be0-48c5-8ae1-716474457f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:45:49.315596 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466ff44-2be0-48c5-8ae1-716474457f16 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:45:49.319393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0c05e-1139-48ce-bee2-a3ebd21d35b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b7a38b4a7754037a00d11dafc88645b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:45:49.320073 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0c05e-1139-48ce-bee2-a3ebd21d35b3 service glance] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:45:49.320394 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0c05e-1139-48ce-bee2-a3ebd21d35b3 service glance] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:45:49.320699 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0c05e-1139-48ce-bee2-a3ebd21d35b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:45:49.329267 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0c05e-1139-48ce-bee2-a3ebd21d35b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:45:49.330792 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0c05e-1139-48ce-bee2-a3ebd21d35b3 service glance] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:45:49.340169 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1434/2855] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:45:49 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:45:59.214115 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=22 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:45:59.335955 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:46:15.031805 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d0699-8930-426a-a6d6-593b8dfa0cc5 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:15.093853 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d0699-8930-426a-a6d6-593b8dfa0cc5 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:15.098444 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d490210-732b-428e-8dd7-83cf0b744862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:15.099340 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d490210-732b-428e-8dd7-83cf0b744862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:15.099621 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d490210-732b-428e-8dd7-83cf0b744862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:15.099892 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d490210-732b-428e-8dd7-83cf0b744862 admin admin] PATH_INFO: `/v3/users/b2a3c0c826294d088634f62dcc00c9bb` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:15.101208 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d490210-732b-428e-8dd7-83cf0b744862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2a3c0c826294d088634f62dcc00c9bb)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:15.104020 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d490210-732b-428e-8dd7-83cf0b744862 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:15.154717 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7d490210-732b-428e-8dd7-83cf0b744862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2a3c0c826294d088634f62dcc00c9bb'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:15.161384 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7d490210-732b-428e-8dd7-83cf0b744862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2a3c0c826294d088634f62dcc00c9bb'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:15.162001 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7d490210-732b-428e-8dd7-83cf0b744862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2a3c0c826294d088634f62dcc00c9bb'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:15.162614 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7d490210-732b-428e-8dd7-83cf0b744862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2a3c0c826294d088634f62dcc00c9bb'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:15.175773 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7d490210-732b-428e-8dd7-83cf0b744862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2a3c0c826294d088634f62dcc00c9bb'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:15.188509 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1422/2856] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:46:15 2022] DELETE /identity/v3/users/b2a3c0c826294d088634f62dcc00c9bb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:46:15.193067 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86648fb-3fb8-4cd1-b9c6-d409c4b69c87 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:15.193557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86648fb-3fb8-4cd1-b9c6-d409c4b69c87 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:15.193711 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86648fb-3fb8-4cd1-b9c6-d409c4b69c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:15.211576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-b86648fb-3fb8-4cd1-b9c6-d409c4b69c87 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:46:15.212950 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b86648fb-3fb8-4cd1-b9c6-d409c4b69c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:46:15.276530 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1435/2857] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 20:46:15.286634 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d490210-732b-428e-8dd7-83cf0b744862 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:15.288112 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7d490210-732b-428e-8dd7-83cf0b744862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:46:15.352985 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d490210-732b-428e-8dd7-83cf0b744862 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:15.357275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd0445-cbc2-4f26-8396-0e940f37a430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:15.357929 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd0445-cbc2-4f26-8396-0e940f37a430 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:15.358209 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd0445-cbc2-4f26-8396-0e940f37a430 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:15.358485 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd0445-cbc2-4f26-8396-0e940f37a430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:15.369458 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd0445-cbc2-4f26-8396-0e940f37a430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:15.371110 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd0445-cbc2-4f26-8396-0e940f37a430 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:15.382850 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1423/2858] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:46:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:46:15.676464 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86648fb-3fb8-4cd1-b9c6-d409c4b69c87 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:15.678191 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b86648fb-3fb8-4cd1-b9c6-d409c4b69c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:46:15.693448 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86648fb-3fb8-4cd1-b9c6-d409c4b69c87 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:15.698284 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24eacf-b9cd-4928-9087-276943d7d79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:15.699195 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24eacf-b9cd-4928-9087-276943d7d79e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:15.699489 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24eacf-b9cd-4928-9087-276943d7d79e admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:15.699762 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24eacf-b9cd-4928-9087-276943d7d79e admin admin] PATH_INFO: `/v3/projects/296bf24530494ecfa32be16d52e6b1b8` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:15.700984 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24eacf-b9cd-4928-9087-276943d7d79e admin admin] RBAC: Authorizing `identity:delete_project(project_id=296bf24530494ecfa32be16d52e6b1b8)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:15.702889 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24eacf-b9cd-4928-9087-276943d7d79e admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:15.758287 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-9a24eacf-b9cd-4928-9087-276943d7d79e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '296bf24530494ecfa32be16d52e6b1b8', 'request_id': 'req-9a24eacf-b9cd-4928-9087-276943d7d79e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:15.767471 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-9a24eacf-b9cd-4928-9087-276943d7d79e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '296bf24530494ecfa32be16d52e6b1b8', 'request_id': 'req-9a24eacf-b9cd-4928-9087-276943d7d79e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:15.774513 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-9a24eacf-b9cd-4928-9087-276943d7d79e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '296bf24530494ecfa32be16d52e6b1b8', 'request_id': 'req-9a24eacf-b9cd-4928-9087-276943d7d79e'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:15.785670 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-9a24eacf-b9cd-4928-9087-276943d7d79e admin admin] The token cache is being invalidate because project 296bf24530494ecfa32be16d52e6b1b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:46:15.785956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-9a24eacf-b9cd-4928-9087-276943d7d79e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:15.786597 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-9a24eacf-b9cd-4928-9087-276943d7d79e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:15.788162 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1436/2859] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:46:15 2022] DELETE /identity/v3/projects/296bf24530494ecfa32be16d52e6b1b8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:46:15.793091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d4800-caba-43f3-8207-b32e3cb62dff None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:15.793423 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d4800-caba-43f3-8207-b32e3cb62dff None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:15.793704 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d4800-caba-43f3-8207-b32e3cb62dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:15.813289 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-dc8d4800-caba-43f3-8207-b32e3cb62dff None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:46:15.814506 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-dc8d4800-caba-43f3-8207-b32e3cb62dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:46:15.832685 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:46:15.877658 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1424/2860] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:46:15.886013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11848d4b-213b-4069-afd4-b0590809690e None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:15.886305 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11848d4b-213b-4069-afd4-b0590809690e None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:15.886589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11848d4b-213b-4069-afd4-b0590809690e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:15.906245 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-11848d4b-213b-4069-afd4-b0590809690e None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:46:15.907493 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-11848d4b-213b-4069-afd4-b0590809690e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:46:15.915391 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1437/2861] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:46:15.919778 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d4800-caba-43f3-8207-b32e3cb62dff None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:15.929645 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d4800-caba-43f3-8207-b32e3cb62dff None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:15.934256 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355672b-6206-42b0-a5ae-ff4c14e34c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:15.934905 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355672b-6206-42b0-a5ae-ff4c14e34c5e admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:15.935210 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355672b-6206-42b0-a5ae-ff4c14e34c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:15.935481 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355672b-6206-42b0-a5ae-ff4c14e34c5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:15.936198 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355672b-6206-42b0-a5ae-ff4c14e34c5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:46:15.936497 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355672b-6206-42b0-a5ae-ff4c14e34c5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:15.938079 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355672b-6206-42b0-a5ae-ff4c14e34c5e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:15.950850 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1425/2862] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:46:15.955281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11848d4b-213b-4069-afd4-b0590809690e None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:15.965257 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11848d4b-213b-4069-afd4-b0590809690e None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:15.970154 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35620f9e-d6ac-4875-9b0d-40351e4835bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:15.970811 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35620f9e-d6ac-4875-9b0d-40351e4835bb admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:15.971119 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35620f9e-d6ac-4875-9b0d-40351e4835bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:15.971397 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35620f9e-d6ac-4875-9b0d-40351e4835bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:15.972106 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35620f9e-d6ac-4875-9b0d-40351e4835bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:15.974012 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35620f9e-d6ac-4875-9b0d-40351e4835bb admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:16.005196 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1438/2863] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:46:15 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:46:16.009370 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355672b-6206-42b0-a5ae-ff4c14e34c5e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:16.020398 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355672b-6206-42b0-a5ae-ff4c14e34c5e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:16.083153 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8fa27-0069-4c58-a885-f801a263a750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:16.083895 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8fa27-0069-4c58-a885-f801a263a750 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:16.084090 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8fa27-0069-4c58-a885-f801a263a750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:16.084365 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8fa27-0069-4c58-a885-f801a263a750 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:16.085063 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc8fa27-0069-4c58-a885-f801a263a750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:16.087013 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc8fa27-0069-4c58-a885-f801a263a750 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:16.108501 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1426/2864] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:46:16 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:46:16.112987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35620f9e-d6ac-4875-9b0d-40351e4835bb admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:16.124125 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35620f9e-d6ac-4875-9b0d-40351e4835bb admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:16.129739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca406982-6626-46d6-8a38-9a27a378fe15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:16.130618 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca406982-6626-46d6-8a38-9a27a378fe15 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:16.130927 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca406982-6626-46d6-8a38-9a27a378fe15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:16.131236 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca406982-6626-46d6-8a38-9a27a378fe15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:16.135355 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca406982-6626-46d6-8a38-9a27a378fe15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:16.135355 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca406982-6626-46d6-8a38-9a27a378fe15 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:16.146464 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1439/2865] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:46:16.150998 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8fa27-0069-4c58-a885-f801a263a750 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:16.161539 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8fa27-0069-4c58-a885-f801a263a750 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:16.166597 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8a49d-ee6b-4a96-a686-e3be69e568d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:16.167312 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8a49d-ee6b-4a96-a686-e3be69e568d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:16.167602 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8a49d-ee6b-4a96-a686-e3be69e568d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:16.167881 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8a49d-ee6b-4a96-a686-e3be69e568d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:16.168648 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8a49d-ee6b-4a96-a686-e3be69e568d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:16.170265 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8a49d-ee6b-4a96-a686-e3be69e568d7 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:16.182275 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1427/2866] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:46:16.187015 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca406982-6626-46d6-8a38-9a27a378fe15 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:16.197913 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca406982-6626-46d6-8a38-9a27a378fe15 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:16.202699 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6389c55-396a-4d48-87c6-c2178c1e8395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:16.203619 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6389c55-396a-4d48-87c6-c2178c1e8395 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:16.203901 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6389c55-396a-4d48-87c6-c2178c1e8395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:16.204177 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6389c55-396a-4d48-87c6-c2178c1e8395 admin admin] PATH_INFO: `/v3/projects/3a079e511080435a8ce560675dc57f4f/users/7ceb333427064dc59795c51ea1c37f3a/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:16.220880 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6389c55-396a-4d48-87c6-c2178c1e8395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a079e511080435a8ce560675dc57f4f, user_id=7ceb333427064dc59795c51ea1c37f3a, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:16.223576 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6389c55-396a-4d48-87c6-c2178c1e8395 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:16.237057 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1440/2867] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:46:16 2022] PUT /identity/v3/projects/3a079e511080435a8ce560675dc57f4f/users/7ceb333427064dc59795c51ea1c37f3a/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:46:16.249279 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eef16d-9e20-4720-9401-13f7bfd896f1 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:16.249576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eef16d-9e20-4720-9401-13f7bfd896f1 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:16.249890 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eef16d-9e20-4720-9401-13f7bfd896f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:16.277638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-09eef16d-9e20-4720-9401-13f7bfd896f1 None None] MFA Rules not processed for user `7ceb333427064dc59795c51ea1c37f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:46:16.278795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-09eef16d-9e20-4720-9401-13f7bfd896f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:46:16.351678 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1428/2868] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:46:16 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:46:16.364476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6389c55-396a-4d48-87c6-c2178c1e8395 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:16.366008 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b6389c55-396a-4d48-87c6-c2178c1e8395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:46:16.391312 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:46:16.427934 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6389c55-396a-4d48-87c6-c2178c1e8395 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:16.432723 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf44bcd-e30c-47e8-a75b-08398c181fca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:16.433386 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf44bcd-e30c-47e8-a75b-08398c181fca service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:16.433673 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf44bcd-e30c-47e8-a75b-08398c181fca service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:16.433951 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf44bcd-e30c-47e8-a75b-08398c181fca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:16.443508 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf44bcd-e30c-47e8-a75b-08398c181fca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:16.445009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf44bcd-e30c-47e8-a75b-08398c181fca service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:16.455944 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1441/2869] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:46:16 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:46:26.348482 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:46:26.450735 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=17 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:46:50.488823 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eef16d-9e20-4720-9401-13f7bfd896f1 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:50.555337 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eef16d-9e20-4720-9401-13f7bfd896f1 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:50.560748 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ed450-99dd-45f9-bedd-3528bcbe0f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:50.561616 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ed450-99dd-45f9-bedd-3528bcbe0f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:50.561903 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ed450-99dd-45f9-bedd-3528bcbe0f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:50.562186 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ed450-99dd-45f9-bedd-3528bcbe0f02 admin admin] PATH_INFO: `/v3/users/7ceb333427064dc59795c51ea1c37f3a` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:50.563402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ed450-99dd-45f9-bedd-3528bcbe0f02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ceb333427064dc59795c51ea1c37f3a)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:50.565326 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ed450-99dd-45f9-bedd-3528bcbe0f02 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:50.610264 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aa9ed450-99dd-45f9-bedd-3528bcbe0f02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ceb333427064dc59795c51ea1c37f3a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:50.617449 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aa9ed450-99dd-45f9-bedd-3528bcbe0f02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ceb333427064dc59795c51ea1c37f3a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:50.618077 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aa9ed450-99dd-45f9-bedd-3528bcbe0f02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ceb333427064dc59795c51ea1c37f3a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:50.618680 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aa9ed450-99dd-45f9-bedd-3528bcbe0f02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ceb333427064dc59795c51ea1c37f3a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:50.631625 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-aa9ed450-99dd-45f9-bedd-3528bcbe0f02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ceb333427064dc59795c51ea1c37f3a'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:50.644736 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1429/2870] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:46:50 2022] DELETE /identity/v3/users/7ceb333427064dc59795c51ea1c37f3a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:46:50.650474 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2545b-f95d-4e23-b48b-b0a68cb3b119 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:50.650987 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2545b-f95d-4e23-b48b-b0a68cb3b119 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:50.651133 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2545b-f95d-4e23-b48b-b0a68cb3b119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:50.669062 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-b5c2545b-f95d-4e23-b48b-b0a68cb3b119 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:46:50.670317 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b5c2545b-f95d-4e23-b48b-b0a68cb3b119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:46:50.734006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1442/2871] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:46:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:46:50.743967 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ed450-99dd-45f9-bedd-3528bcbe0f02 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:50.745894 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-aa9ed450-99dd-45f9-bedd-3528bcbe0f02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:46:50.802659 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ed450-99dd-45f9-bedd-3528bcbe0f02 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:50.806511 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f6016-3c43-4bf1-82d5-eb5a49557a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:50.807671 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f6016-3c43-4bf1-82d5-eb5a49557a86 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:50.807961 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f6016-3c43-4bf1-82d5-eb5a49557a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:50.808236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f6016-3c43-4bf1-82d5-eb5a49557a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:50.817460 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f6016-3c43-4bf1-82d5-eb5a49557a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:50.819180 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f6016-3c43-4bf1-82d5-eb5a49557a86 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:50.829833 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1430/2872] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:46:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:46:51.109451 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2545b-f95d-4e23-b48b-b0a68cb3b119 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:51.111006 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b5c2545b-f95d-4e23-b48b-b0a68cb3b119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:46:51.123738 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2545b-f95d-4e23-b48b-b0a68cb3b119 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:51.128416 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da45fd-cc59-4db0-9c25-479ddd8a0d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:51.129287 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da45fd-cc59-4db0-9c25-479ddd8a0d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:51.129568 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da45fd-cc59-4db0-9c25-479ddd8a0d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:51.129850 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da45fd-cc59-4db0-9c25-479ddd8a0d77 admin admin] PATH_INFO: `/v3/projects/3a079e511080435a8ce560675dc57f4f` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:51.131159 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35da45fd-cc59-4db0-9c25-479ddd8a0d77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a079e511080435a8ce560675dc57f4f)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:51.132989 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35da45fd-cc59-4db0-9c25-479ddd8a0d77 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:51.181058 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-35da45fd-cc59-4db0-9c25-479ddd8a0d77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a079e511080435a8ce560675dc57f4f', 'request_id': 'req-35da45fd-cc59-4db0-9c25-479ddd8a0d77'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:51.189121 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-35da45fd-cc59-4db0-9c25-479ddd8a0d77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a079e511080435a8ce560675dc57f4f', 'request_id': 'req-35da45fd-cc59-4db0-9c25-479ddd8a0d77'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:51.195347 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-35da45fd-cc59-4db0-9c25-479ddd8a0d77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a079e511080435a8ce560675dc57f4f', 'request_id': 'req-35da45fd-cc59-4db0-9c25-479ddd8a0d77'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:51.205739 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-35da45fd-cc59-4db0-9c25-479ddd8a0d77 admin admin] The token cache is being invalidate because project 3a079e511080435a8ce560675dc57f4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:46:51.206030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-35da45fd-cc59-4db0-9c25-479ddd8a0d77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:51.206664 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-35da45fd-cc59-4db0-9c25-479ddd8a0d77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:51.208234 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1443/2873] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:46:51 2022] DELETE /identity/v3/projects/3a079e511080435a8ce560675dc57f4f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:46:51.213188 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d6789-b1b1-4529-b330-3e2f69f3392f None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:51.213934 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d6789-b1b1-4529-b330-3e2f69f3392f None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:51.214220 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d6789-b1b1-4529-b330-3e2f69f3392f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:51.232445 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-f13d6789-b1b1-4529-b330-3e2f69f3392f None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:46:51.233647 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-f13d6789-b1b1-4529-b330-3e2f69f3392f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:46:51.251412 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:46:51.295238 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1431/2874] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:46:51.304258 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b1c25-379e-4fb8-beab-3da1fbebe116 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:51.304720 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b1c25-379e-4fb8-beab-3da1fbebe116 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:51.305084 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b1c25-379e-4fb8-beab-3da1fbebe116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:51.324697 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-245b1c25-379e-4fb8-beab-3da1fbebe116 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:46:51.325911 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-245b1c25-379e-4fb8-beab-3da1fbebe116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:46:51.333950 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1444/2875] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:46:51.338261 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d6789-b1b1-4529-b330-3e2f69f3392f None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:51.348505 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d6789-b1b1-4529-b330-3e2f69f3392f None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:51.353033 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283f080-5fcc-4902-ac6f-18713e2e7373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:51.353707 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283f080-5fcc-4902-ac6f-18713e2e7373 admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:51.353986 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283f080-5fcc-4902-ac6f-18713e2e7373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:51.354849 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283f080-5fcc-4902-ac6f-18713e2e7373 admin admin] PATH_INFO: `/v3/domains` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:51.355700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283f080-5fcc-4902-ac6f-18713e2e7373 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80177) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 20:46:51.356002 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283f080-5fcc-4902-ac6f-18713e2e7373 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:51.357576 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283f080-5fcc-4902-ac6f-18713e2e7373 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:51.371710 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1432/2876] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 14 20:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 20:46:51.376044 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b1c25-379e-4fb8-beab-3da1fbebe116 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:51.385436 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b1c25-379e-4fb8-beab-3da1fbebe116 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:51.390152 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a8a74-67c0-4590-99d5-b3957dc0097e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:51.390800 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a8a74-67c0-4590-99d5-b3957dc0097e admin admin] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:51.391127 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a8a74-67c0-4590-99d5-b3957dc0097e admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:51.391409 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a8a74-67c0-4590-99d5-b3957dc0097e admin admin] PATH_INFO: `/v3/projects` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:51.392107 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a8a74-67c0-4590-99d5-b3957dc0097e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:51.394016 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a8a74-67c0-4590-99d5-b3957dc0097e admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:51.421621 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1445/2877] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 14 20:46:51 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:46:51.425809 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283f080-5fcc-4902-ac6f-18713e2e7373 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:51.435743 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283f080-5fcc-4902-ac6f-18713e2e7373 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:51.493053 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8335b3-266e-4c8b-bd1b-642f3675bdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:51.493791 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8335b3-266e-4c8b-bd1b-642f3675bdd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:51.493989 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8335b3-266e-4c8b-bd1b-642f3675bdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:51.494265 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8335b3-266e-4c8b-bd1b-642f3675bdd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:51.494983 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8335b3-266e-4c8b-bd1b-642f3675bdd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:51.496896 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8335b3-266e-4c8b-bd1b-642f3675bdd2 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:51.516522 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1433/2878] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 14 20:46:51 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 20:46:51.520943 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a8a74-67c0-4590-99d5-b3957dc0097e admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:51.532020 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a8a74-67c0-4590-99d5-b3957dc0097e admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:51.537007 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba3229-9c31-485a-933f-27777f14d9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:51.537774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba3229-9c31-485a-933f-27777f14d9c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:51.537951 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba3229-9c31-485a-933f-27777f14d9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:51.538228 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba3229-9c31-485a-933f-27777f14d9c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:51.539055 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba3229-9c31-485a-933f-27777f14d9c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:51.540652 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba3229-9c31-485a-933f-27777f14d9c9 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:51.552118 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1446/2879] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:46:51.556245 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8335b3-266e-4c8b-bd1b-642f3675bdd2 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:51.567526 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8335b3-266e-4c8b-bd1b-642f3675bdd2 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:51.572811 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d0bfd-1b0d-446b-8cd6-d0f34ee6913b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:51.573468 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d0bfd-1b0d-446b-8cd6-d0f34ee6913b admin admin] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:51.573861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d0bfd-1b0d-446b-8cd6-d0f34ee6913b admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:51.574164 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d0bfd-1b0d-446b-8cd6-d0f34ee6913b admin admin] PATH_INFO: `/v3/roles` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:51.574930 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d0bfd-1b0d-446b-8cd6-d0f34ee6913b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:51.576507 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d0bfd-1b0d-446b-8cd6-d0f34ee6913b admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:51.586497 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1434/2880] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 14 20:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 20:46:51.591209 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba3229-9c31-485a-933f-27777f14d9c9 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:51.601144 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba3229-9c31-485a-933f-27777f14d9c9 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:51.606302 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe74ea-173f-433a-94ca-5bff4f07f3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:51.607220 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe74ea-173f-433a-94ca-5bff4f07f3c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:51.607503 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe74ea-173f-433a-94ca-5bff4f07f3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:51.607779 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe74ea-173f-433a-94ca-5bff4f07f3c2 admin admin] PATH_INFO: `/v3/projects/c13db2d9b5cd46328d3fcd8c3616b902/users/6ea6a71774514b7497cc01900a402f51/roles/b778e530d54948c7bc31f5b9231ea288` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:51.622256 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe74ea-173f-433a-94ca-5bff4f07f3c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c13db2d9b5cd46328d3fcd8c3616b902, user_id=6ea6a71774514b7497cc01900a402f51, role_id=b778e530d54948c7bc31f5b9231ea288)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:51.624906 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe74ea-173f-433a-94ca-5bff4f07f3c2 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:51.638844 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1447/2881] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 14 20:46:51 2022] PUT /identity/v3/projects/c13db2d9b5cd46328d3fcd8c3616b902/users/6ea6a71774514b7497cc01900a402f51/roles/b778e530d54948c7bc31f5b9231ea288 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:46:51.650236 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b790e98-920a-4438-877f-8d9d49999020 None None] REQUEST_METHOD: `POST` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:51.650522 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b790e98-920a-4438-877f-8d9d49999020 None None] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:51.650797 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b790e98-920a-4438-877f-8d9d49999020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:51.684318 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.auth.core [None req-4b790e98-920a-4438-877f-8d9d49999020 None None] MFA Rules not processed for user `6ea6a71774514b7497cc01900a402f51`. Rule list: `[]` (Enabled: `True`). {{(pid=80177) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:46:51.685795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-4b790e98-920a-4438-877f-8d9d49999020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:46:51.756125 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1435/2882] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:46:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:46:51.768835 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe74ea-173f-433a-94ca-5bff4f07f3c2 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:51.770807 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-4fbe74ea-173f-433a-94ca-5bff4f07f3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:46:51.792890 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:46:51.837298 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe74ea-173f-433a-94ca-5bff4f07f3c2 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:51.840904 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f7b00-3b52-4936-8550-b459ca320456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b4142ea6f142df9b1314532c961f75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:51.841545 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f7b00-3b52-4936-8550-b459ca320456 service cinder] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:51.841825 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f7b00-3b52-4936-8550-b459ca320456 service cinder] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:51.842099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f7b00-3b52-4936-8550-b459ca320456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:51.850281 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f7b00-3b52-4936-8550-b459ca320456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:51.851744 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f7b00-3b52-4936-8550-b459ca320456 service cinder] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:51.861013 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1448/2883] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:46:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:46:51.885785 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b790e98-920a-4438-877f-8d9d49999020 None None] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:51.905036 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:46:51.951746 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b790e98-920a-4438-877f-8d9d49999020 None None] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:51.956402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3284c-65fe-49f3-8b17-42aea1f3443e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:51.957274 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3284c-65fe-49f3-8b17-42aea1f3443e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:51.957564 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3284c-65fe-49f3-8b17-42aea1f3443e admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:51.957841 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3284c-65fe-49f3-8b17-42aea1f3443e admin admin] PATH_INFO: `/v3/users/6ea6a71774514b7497cc01900a402f51` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:51.959075 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3284c-65fe-49f3-8b17-42aea1f3443e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ea6a71774514b7497cc01900a402f51)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:51.960995 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3284c-65fe-49f3-8b17-42aea1f3443e admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:52.009853 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7dd3284c-65fe-49f3-8b17-42aea1f3443e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ea6a71774514b7497cc01900a402f51'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:52.017383 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7dd3284c-65fe-49f3-8b17-42aea1f3443e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ea6a71774514b7497cc01900a402f51'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:52.018205 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7dd3284c-65fe-49f3-8b17-42aea1f3443e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ea6a71774514b7497cc01900a402f51'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:52.018795 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7dd3284c-65fe-49f3-8b17-42aea1f3443e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ea6a71774514b7497cc01900a402f51'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:52.031389 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-7dd3284c-65fe-49f3-8b17-42aea1f3443e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ea6a71774514b7497cc01900a402f51'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:52.043831 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1436/2884] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:46:51 2022] DELETE /identity/v3/users/6ea6a71774514b7497cc01900a402f51 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:46:52.048358 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5622820-0b86-49f5-97a9-966825852f88 None None] REQUEST_METHOD: `POST` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:52.048655 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5622820-0b86-49f5-97a9-966825852f88 None None] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:52.048970 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5622820-0b86-49f5-97a9-966825852f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:52.065956 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.auth.core [None req-d5622820-0b86-49f5-97a9-966825852f88 None None] MFA Rules not processed for user `8b5c1280f1184855bd78412238939419`. Rule list: `[]` (Enabled: `True`). {{(pid=80178) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 20:46:52.067228 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d5622820-0b86-49f5-97a9-966825852f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:46:52.130358 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1449/2885] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 14 20:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 20:46:52.141029 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3284c-65fe-49f3-8b17-42aea1f3443e admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:52.142488 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-7dd3284c-65fe-49f3-8b17-42aea1f3443e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:46:52.205950 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3284c-65fe-49f3-8b17-42aea1f3443e admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:52.210058 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131e3dc-49b2-47b7-8b2c-225539fd74b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:52.210700 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131e3dc-49b2-47b7-8b2c-225539fd74b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:52.211010 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131e3dc-49b2-47b7-8b2c-225539fd74b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:52.211282 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131e3dc-49b2-47b7-8b2c-225539fd74b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:52.221861 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131e3dc-49b2-47b7-8b2c-225539fd74b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:52.223324 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131e3dc-49b2-47b7-8b2c-225539fd74b3 service neutron] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:52.233553 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1437/2886] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:46:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:46:52.517157 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5622820-0b86-49f5-97a9-966825852f88 None None] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:46:52.519048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-d5622820-0b86-49f5-97a9-966825852f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:46:52.533205 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5622820-0b86-49f5-97a9-966825852f88 None None] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:46:52.538874 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f33dd-a837-4abd-8313-10ca5a775a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:46:52.539791 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f33dd-a837-4abd-8313-10ca5a775a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:46:52.540077 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f33dd-a837-4abd-8313-10ca5a775a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:46:52.540353 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f33dd-a837-4abd-8313-10ca5a775a92 admin admin] PATH_INFO: `/v3/projects/c13db2d9b5cd46328d3fcd8c3616b902` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:46:52.541914 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f33dd-a837-4abd-8313-10ca5a775a92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c13db2d9b5cd46328d3fcd8c3616b902)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:46:52.543543 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f33dd-a837-4abd-8313-10ca5a775a92 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:46:52.592325 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-b59f33dd-a837-4abd-8313-10ca5a775a92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c13db2d9b5cd46328d3fcd8c3616b902', 'request_id': 'req-b59f33dd-a837-4abd-8313-10ca5a775a92'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:52.600207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-b59f33dd-a837-4abd-8313-10ca5a775a92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c13db2d9b5cd46328d3fcd8c3616b902', 'request_id': 'req-b59f33dd-a837-4abd-8313-10ca5a775a92'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:52.609717 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-b59f33dd-a837-4abd-8313-10ca5a775a92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c13db2d9b5cd46328d3fcd8c3616b902', 'request_id': 'req-b59f33dd-a837-4abd-8313-10ca5a775a92'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:52.620345 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-b59f33dd-a837-4abd-8313-10ca5a775a92 admin admin] The token cache is being invalidate because project c13db2d9b5cd46328d3fcd8c3616b902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80178) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:46:52.620633 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-b59f33dd-a837-4abd-8313-10ca5a775a92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:52.621282 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-b59f33dd-a837-4abd-8313-10ca5a775a92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:46:52.622848 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1450/2887] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:46:52 2022] DELETE /identity/v3/projects/c13db2d9b5cd46328d3fcd8c3616b902 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:47:02.228724 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:47:02.617265 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:47:10.869425 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131e3dc-49b2-47b7-8b2c-225539fd74b3 service neutron] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:47:10.872285 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a131e3dc-49b2-47b7-8b2c-225539fd74b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:47:10.979091 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131e3dc-49b2-47b7-8b2c-225539fd74b3 service neutron] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:47:11.079758 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c4bc4-a928-4d43-b3b7-00a04bdd6b90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa3bb9f1e24a4e9ebd460e97b3736bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:47:11.080360 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c4bc4-a928-4d43-b3b7-00a04bdd6b90 service nova] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:47:11.080639 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c4bc4-a928-4d43-b3b7-00a04bdd6b90 service nova] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:47:11.080920 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c4bc4-a928-4d43-b3b7-00a04bdd6b90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:47:11.082827 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-575c4bc4-a928-4d43-b3b7-00a04bdd6b90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:47:11.097127 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c4bc4-a928-4d43-b3b7-00a04bdd6b90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:47:11.098082 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c4bc4-a928-4d43-b3b7-00a04bdd6b90 service nova] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:47:11.111677 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1438/2888] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:47:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:47:14.803703 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f33dd-a837-4abd-8313-10ca5a775a92 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:47:14.805207 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-b59f33dd-a837-4abd-8313-10ca5a775a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:47:14.874492 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f33dd-a837-4abd-8313-10ca5a775a92 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:47:14.878105 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771d6dc-0471-4fad-bb17-aec473c22faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f88257850ee34ad895208ab935b6e643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:47:14.878818 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771d6dc-0471-4fad-bb17-aec473c22faf service neutron] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:47:14.879219 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771d6dc-0471-4fad-bb17-aec473c22faf service neutron] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:47:14.879507 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771d6dc-0471-4fad-bb17-aec473c22faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:47:14.881589 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9771d6dc-0471-4fad-bb17-aec473c22faf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:47:14.950557 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771d6dc-0471-4fad-bb17-aec473c22faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:47:14.952030 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771d6dc-0471-4fad-bb17-aec473c22faf service neutron] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:47:14.966514 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1451/2889] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:47:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:47:18.232045 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c4bc4-a928-4d43-b3b7-00a04bdd6b90 service nova] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:47:18.233531 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-575c4bc4-a928-4d43-b3b7-00a04bdd6b90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:47:18.322790 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c4bc4-a928-4d43-b3b7-00a04bdd6b90 service nova] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:47:18.328305 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756584a1-30e0-4c3f-9805-61729538d91e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afad68819504c2b864851c6989ae25a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:47:18.328957 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756584a1-30e0-4c3f-9805-61729538d91e service placement] REQUEST_METHOD: `GET` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:47:18.329235 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756584a1-30e0-4c3f-9805-61729538d91e service placement] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:47:18.329516 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756584a1-30e0-4c3f-9805-61729538d91e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:47:18.332192 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-756584a1-30e0-4c3f-9805-61729538d91e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:47:18.353477 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756584a1-30e0-4c3f-9805-61729538d91e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:47:18.356568 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756584a1-30e0-4c3f-9805-61729538d91e service placement] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:47:18.375892 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1439/2890] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:47:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 20:47:21.686309 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771d6dc-0471-4fad-bb17-aec473c22faf service neutron] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:47:21.688099 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-9771d6dc-0471-4fad-bb17-aec473c22faf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:47:21.705992 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771d6dc-0471-4fad-bb17-aec473c22faf service neutron] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:47:21.711294 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7a04a-6b7e-474e-a1e6-66555dc6b465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:47:21.712186 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7a04a-6b7e-474e-a1e6-66555dc6b465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:47:21.712476 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7a04a-6b7e-474e-a1e6-66555dc6b465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:47:21.712758 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7a04a-6b7e-474e-a1e6-66555dc6b465 admin admin] PATH_INFO: `/v3/users/c0d43152eeb9485e9ff16f38aa63ac22` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:47:21.714009 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7a04a-6b7e-474e-a1e6-66555dc6b465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0d43152eeb9485e9ff16f38aa63ac22)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:47:21.716017 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7a04a-6b7e-474e-a1e6-66555dc6b465 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:47:21.774460 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-edb7a04a-6b7e-474e-a1e6-66555dc6b465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0d43152eeb9485e9ff16f38aa63ac22'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:21.784459 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-edb7a04a-6b7e-474e-a1e6-66555dc6b465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0d43152eeb9485e9ff16f38aa63ac22'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:21.793128 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-edb7a04a-6b7e-474e-a1e6-66555dc6b465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0d43152eeb9485e9ff16f38aa63ac22'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:21.793862 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-edb7a04a-6b7e-474e-a1e6-66555dc6b465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0d43152eeb9485e9ff16f38aa63ac22'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:21.794469 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-edb7a04a-6b7e-474e-a1e6-66555dc6b465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0d43152eeb9485e9ff16f38aa63ac22'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:21.813709 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1452/2891] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:47:21 2022] DELETE /identity/v3/users/c0d43152eeb9485e9ff16f38aa63ac22 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:47:22.007884 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756584a1-30e0-4c3f-9805-61729538d91e service placement] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:47:22.020911 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756584a1-30e0-4c3f-9805-61729538d91e service placement] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:47:22.032324 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=100 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:47:22.087847 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806cdb5-6e1a-4766-ada8-dae7af6eb914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:47:22.088638 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806cdb5-6e1a-4766-ada8-dae7af6eb914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:47:22.088921 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806cdb5-6e1a-4766-ada8-dae7af6eb914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:47:22.089198 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806cdb5-6e1a-4766-ada8-dae7af6eb914 admin admin] PATH_INFO: `/v3/projects/c8ded6a71283444e892094501f29e4da` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:47:22.090533 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806cdb5-6e1a-4766-ada8-dae7af6eb914 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8ded6a71283444e892094501f29e4da)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:47:22.092402 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806cdb5-6e1a-4766-ada8-dae7af6eb914 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:47:22.144755 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a806cdb5-6e1a-4766-ada8-dae7af6eb914 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8ded6a71283444e892094501f29e4da', 'request_id': 'req-a806cdb5-6e1a-4766-ada8-dae7af6eb914'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:22.152767 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a806cdb5-6e1a-4766-ada8-dae7af6eb914 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8ded6a71283444e892094501f29e4da', 'request_id': 'req-a806cdb5-6e1a-4766-ada8-dae7af6eb914'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:22.161285 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a806cdb5-6e1a-4766-ada8-dae7af6eb914 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8ded6a71283444e892094501f29e4da', 'request_id': 'req-a806cdb5-6e1a-4766-ada8-dae7af6eb914'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:22.186163 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a806cdb5-6e1a-4766-ada8-dae7af6eb914 admin admin] The token cache is being invalidate because project c8ded6a71283444e892094501f29e4da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:47:22.186470 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a806cdb5-6e1a-4766-ada8-dae7af6eb914 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:22.187489 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-a806cdb5-6e1a-4766-ada8-dae7af6eb914 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:22.192467 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1440/2892] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:47:22 2022] DELETE /identity/v3/projects/c8ded6a71283444e892094501f29e4da => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:47:22.193774 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7a04a-6b7e-474e-a1e6-66555dc6b465 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:47:22.195346 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-edb7a04a-6b7e-474e-a1e6-66555dc6b465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:47:22.209381 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7a04a-6b7e-474e-a1e6-66555dc6b465 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:47:22.230585 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:47:22.274442 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d43e77-c457-4e6b-b579-6dacced12a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:47:22.276120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d43e77-c457-4e6b-b579-6dacced12a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:47:22.276120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d43e77-c457-4e6b-b579-6dacced12a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:47:22.276120 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d43e77-c457-4e6b-b579-6dacced12a49 admin admin] PATH_INFO: `/v3/users/aab62a5b981d4aadbd48cd751c5bb529` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:47:22.277418 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d43e77-c457-4e6b-b579-6dacced12a49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aab62a5b981d4aadbd48cd751c5bb529)` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:47:22.279393 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d43e77-c457-4e6b-b579-6dacced12a49 admin admin] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:47:22.332663 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-71d43e77-c457-4e6b-b579-6dacced12a49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aab62a5b981d4aadbd48cd751c5bb529'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:22.340526 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-71d43e77-c457-4e6b-b579-6dacced12a49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aab62a5b981d4aadbd48cd751c5bb529'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:22.348341 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-71d43e77-c457-4e6b-b579-6dacced12a49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aab62a5b981d4aadbd48cd751c5bb529'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:22.349040 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-71d43e77-c457-4e6b-b579-6dacced12a49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aab62a5b981d4aadbd48cd751c5bb529'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:22.349642 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.notifications [None req-71d43e77-c457-4e6b-b579-6dacced12a49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aab62a5b981d4aadbd48cd751c5bb529'} {{(pid=80178) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:22.366925 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1453/2893] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 14 20:47:22 2022] DELETE /identity/v3/users/aab62a5b981d4aadbd48cd751c5bb529 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:47:22.548992 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806cdb5-6e1a-4766-ada8-dae7af6eb914 admin admin] Authenticating user token {{(pid=80177) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:47:22.550535 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.fernet_utils [None req-a806cdb5-6e1a-4766-ada8-dae7af6eb914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80177) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:47:22.563987 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806cdb5-6e1a-4766-ada8-dae7af6eb914 admin admin] Validating token access rules against request {{(pid=80177) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:47:22.626738 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f3afc-7002-4690-85f6-679a0adecb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b5c1280f1184855bd78412238939419', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9054cd3a81847fdaa8f38a717dc0938', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80177) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:47:22.627554 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f3afc-7002-4690-85f6-679a0adecb28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:47:22.627836 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f3afc-7002-4690-85f6-679a0adecb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:47:22.628111 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f3afc-7002-4690-85f6-679a0adecb28 admin admin] PATH_INFO: `/v3/projects/47ebe372bc944ef7967dca5e2ba857f1` {{(pid=80177) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:47:22.629316 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f3afc-7002-4690-85f6-679a0adecb28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47ebe372bc944ef7967dca5e2ba857f1)` {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:47:22.635627 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f3afc-7002-4690-85f6-679a0adecb28 admin admin] RBAC: Authorization granted {{(pid=80177) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:47:22.678649 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:47:22.681867 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-935f3afc-7002-4690-85f6-679a0adecb28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47ebe372bc944ef7967dca5e2ba857f1', 'request_id': 'req-935f3afc-7002-4690-85f6-679a0adecb28'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:22.690411 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-935f3afc-7002-4690-85f6-679a0adecb28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47ebe372bc944ef7967dca5e2ba857f1', 'request_id': 'req-935f3afc-7002-4690-85f6-679a0adecb28'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:22.697253 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-935f3afc-7002-4690-85f6-679a0adecb28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47ebe372bc944ef7967dca5e2ba857f1', 'request_id': 'req-935f3afc-7002-4690-85f6-679a0adecb28'} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:22.711944 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-935f3afc-7002-4690-85f6-679a0adecb28 admin admin] The token cache is being invalidate because project 47ebe372bc944ef7967dca5e2ba857f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80177) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 20:47:22.712229 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-935f3afc-7002-4690-85f6-679a0adecb28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:22.712842 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG keystone.notifications [None req-935f3afc-7002-4690-85f6-679a0adecb28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80177) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 20:47:22.714383 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: [pid: 80177|app: 0|req: 1441/2894] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 14 20:47:22 2022] DELETE /identity/v3/projects/47ebe372bc944ef7967dca5e2ba857f1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 20:47:32.363302 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:47:32.708984 dv-cinder-799871-32-iscsi devstack@keystone.service[80177]: DEBUG dbcounter [-] [80177] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80177) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 20:48:46.850748 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d43e77-c457-4e6b-b579-6dacced12a49 admin admin] Authenticating user token {{(pid=80178) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 20:48:46.852227 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-71d43e77-c457-4e6b-b579-6dacced12a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:48:46.920742 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d43e77-c457-4e6b-b579-6dacced12a49 admin admin] Validating token access rules against request {{(pid=80178) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 20:48:46.978464 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db353c3-88f3-4c30-8b27-4b0ec142679e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afad68819504c2b864851c6989ae25a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f33c8ac5154f46a89d1a0dc11fb58cf4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80178) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 20:48:46.979149 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db353c3-88f3-4c30-8b27-4b0ec142679e service placement] REQUEST_METHOD: `GET` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 20:48:46.979439 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db353c3-88f3-4c30-8b27-4b0ec142679e service placement] SCRIPT_NAME: `/identity` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 20:48:46.979716 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db353c3-88f3-4c30-8b27-4b0ec142679e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80178) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 20:48:46.982051 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.fernet_utils [None req-3db353c3-88f3-4c30-8b27-4b0ec142679e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80178) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 20:48:46.993048 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db353c3-88f3-4c30-8b27-4b0ec142679e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 20:48:46.994603 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db353c3-88f3-4c30-8b27-4b0ec142679e service placement] RBAC: Authorization granted {{(pid=80178) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 20:48:47.004996 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: [pid: 80178|app: 0|req: 1454/2895] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 14 20:48:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 20:48:57.000788 dv-cinder-799871-32-iscsi devstack@keystone.service[80178]: DEBUG dbcounter [-] [80178] Writing DB stats keystone:SELECT=58 {{(pid=80178) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}